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Executive  Summary 


The  process  used  by  the  Department  of  Defense  to  decide  whether  the  public 
(organic)  or  private  (commercial)  sector  should  perform  depot-level  maintenance 
has  proven  less  than  satisfactory.  Both  internal  and  external  critics  continue  to  call 
into  question  the  comprehensiveness,  consistency,  and  fairness  of  the  process,  as 
well  as  its  effectiveness  in  mitigating  performance  risk.  To  some  extent  these 
critical  concerns  are  legitimate;  to  some  extent  they  reflect  the  parochial  interests 
of  the  competing  sectors’  interest  in  retaining  or  obtaining  workload.  Regardless, 
the  department  does  not  have  a  process  for  making  repair  sector  selection  (RSS) 
decisions  in  which  the  various  constituencies  can  place  confidence. 

Contributing  to  the  present  dilemma  is  the  fact  that  historical  rules  of  thumb — for 
example,  defaulting  to  organic  sources  of  repair  in  the  absence  of  some  compel¬ 
ling  reasons  not  to — have  become  suspect  or  controversial  or  both.  There  are  a 
number  of  reasons  for  this  state  of  affairs.  First,  large-scale  reductions  in  the  cost 
of  transportation  and  information  relative  to  other  factors  such  as  inventory  and 
labor  have  helped  fuel  a  well-publicized  shift  to  outsourcing  in  the  commercial 
sector.  Given  the  perceived  benefits,  it  is  small  wonder  that  some  senior  DoD  ex¬ 
ecutives  are  pushing  for  greater  outsourcing  (i.e.,  privatization)  of  depot  mainte¬ 
nance.  Second,  in  the  face  of  a  decline  in  the  acquisition  of  new  weapon  systems, 
original  equipment  manufacturers  and  their  major  subcontractors  have  shown  a 
much  stronger  interest  in  this  work  than  has  historically  been  the  case.  Third, 
third-party  maintenance  providers  have  emerged  with  technical  capabilities  gener¬ 
ally  equivalent  to  those  of  the  public  depots.  And  fourth,  the  total  amount  of  depot 
maintenance  work  has  decreased  since  the  end  of  the  Cold  War,  with  the  result 
that  the  public  and  private  sectors  are  vigorously  contending  for  slices  of  a  dimin¬ 
ishing  pie. 

DoD  has,  at  various  times,  used  methods  such  as  decision  tree  analysis  (DTA), 
life-cycle  cost  analysis  (LCC)  and,  most  recently,  the  CORE  methodology  (a 
process  for  determining  what  workloads  need  to  be  part  of  DoD’s  core  mainte¬ 
nance  capability)  to  make  RSS  decisions.  None  has  proved  satisfactory.  The  DTA 
method,  which  depended  on  simple  yes-no  responses  to  a  series  of  questions,  was 
an  unsophisticated  approach  to  a  complex  problem  that  inherently  lacks  simple 
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answers.  LCC  analysis,  by  focusing  exclusively  on  cost,  ignores  potentially  cru¬ 
cial  differences  in  relative  performance  risk.  Although  the  current  CORE  method¬ 
ology  addresses  risk  and  initially  appeared  to  hold  promise,  in  practice  it  has 
proved  difficult  to  decide  in  any  consistent  way  what  needs  to  be  in  CORE  and 
what  does  not.  And,  in  general,  all  of  the  methods  have  narrowly  focused  on  depot 
maintenance  per  se  rather  than  considering  depot  maintenance  as  one  component 
of  a  supply  chain  extending  from  factory  to  military  end  user. 

To  enable  effective  repair  sector  decisions,  DoD  must,  first,  base  its  decision 
process  on  a  comprehensive  understanding  (i.e.,  a  model)  of  the  key  elements  in 
the  decision  and,  second,  apply  that  understanding  fairly  and  consistently  across 
the  military  services  and  over  time. 

We  developed  and  recommend  for  use  a  model  of  the  RSS  decision  that  incorpo¬ 
rates  the  following: 

♦  Needs  of  the  end-users  of  depot  repair  services  (e.g.,  timeliness,  low  cost, 
quality) 

♦  Characteristics  of  the  two  repair  sectors  such  as  adequate  capability  and 
capacity,  scope  and  scale  economy,  and  responsiveness  that  affect  their 
ability  to  satisfy  user  needs 

♦  Factors  that  determine  the  characteristics  of  the  repair  sectors.  Such  factors 
naturally  group  into  three  categories,  based  on  whether  they 

>-  are  intrinsic  to  the  nature  of  the  repair  sources  themselves, 

►  depend  on  the  relationship  between  the  repair  source  and  the  buyer  of 
repair  services,  or 

>-  are  a  function  of  the  repair  work  to  be  accomplished. 

We  identified  22  individual  factors  that  are  important  to  the  repair  sector  selection 
decision  and,  ultimately,  to  the  risk  associated  with  satisfying  end-user  needs.  Be¬ 
cause  there  are  a  significant  number  of  important  factors  and  none  lend  them¬ 
selves  to  simple  yes-no  answers,  evaluation  is  too  complex  for  a  decision  tree.  To 
deal  with  this  complexity,  we  developed  an  arithmetic  modeling  approach  that 
combines  individual  factor  evaluations  into  figures  of  merit  that  indicate  the  rela¬ 
tive  attractiveness  (or  risk)  of  the  organic  and  private  sectors. 

As  noted  above,  the  second  key  ingredient  of  an  effective  RSS  decision  process  is 
a  means  to  apply  the  decision  model  fairly  and  consistently.  Such  a  mechanism 
must  meet  two  conditions.  The  first  is  that  whoever  does  RSS  evaluations  has  the 
needed  training  and  experience  to  do  them  well.  The  second  is  that  those  who  do 
evaluations  are,  and  are  perceived  as  being,  free  of  conflicts  of  interest  such  as  a 
personal  or  organizational  stake  in  the  outcome. 
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These  two  conditions  are  not  met  today.  With  regard  to  experience  and  training,  as 
an  example,  initial  RSS  decisions  are  typically  made  by  the  staffs  of  system  pro¬ 
gram  offices.  These  individuals  usually  lack  prior  experience  and  training  in  such 
decisions.  Even  where  there  is  prior  experience,  the  lack  of  conflict  provision 
generally  cannot  be  satisfied.  To  solve  these  problems,  we  recommend  the  crea¬ 
tion  of  an  RSS  decision  support  cadre  reporting  to  the  Defense  Depot  Mainte¬ 
nance  Council.  This  cadre  should 

♦  be  truly  impartial  regarding  the  outcome  of  RSS  decisions  and  able  to 
maintain  the  confidence  of  the  competing  constituencies  as  well  as  of  in¬ 
ternal  and  external  process  or  decision  critics; 

♦  have  the  requisite  education,  technical  training,  market  awareness,  and 
policy  perspective — in  particular,  they  should  be  sufficiently  senior  to  be 
on  a  peer  level  with  their  organic  depot  and  commercial  industry  counter¬ 
parts;  and 

♦  be  constituted  to  assist  the  services  in  reaching  depot  maintenance  repair 
sector  selection  decisions  rather  than  taking  the  decisions  out  of  their  hands. 

The  RSS  decision  issue  arises  for  both  new  workloads,  such  as  those  associated 
with  new  weapon  systems  and  major  modifications,  and  for  existing  workloads. 
New  weapon  systems  and  major  modifications  provide  greater  freedom  of  action, 
especially  from  a  political  perspective,  than  do  existing  workloads.  For  this  rea¬ 
son,  we  recommend  that  our  proposed  approach  be  first  applied  to  new  workloads. 
Experience  there  will  provide  a  basis  for  deciding  how  and  when  it  should  be  ap¬ 
plied  to  existing  workloads. 

In  summary,  our  recommendations  are  as  follows: 

♦  A  broader,  balanced  view  of  the  repair  sector  selection  decision  will  ad¬ 
dress  the  need  for  consistency  and  comprehensiveness. 

Incorporate  the  22  critical  factors  into  policy  on  the  RSS  decision  process. 

♦  The  use  of  arithmetic  modeling  in  lieu  of  decision  trees  will  provide  a  de¬ 
cision  support  mechanism  equal  to  the  complexity  of  the  RSS  decision. 

Adopt  an  arithmetic  modeling  approach  to  support  RSS  deliberations. 

♦  The  use  of  a  professional,  impartial  group  to  support  RSS  decisions  will 
mitigate  concerns  about  fairness  and  adequate  consideration  of  risk. 

Establish  a  cadre  of  government  professionals  to  carry  out  RSS  analyses 
in  support  of  all  the  military  services.  Consider  a  special  cell  reporting  to 
the  Defense  Depot  Maintenance  Council. 
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Chapter  1 

Introduction 


Purpose  and  Scope 

The  purpose  of  this  study  was  to  reexamine  the  Department  of  Defense  process 
for  deciding  whether  the  public  (organic)  or  private  (commercial)  sector  should 
perform  depot-level  maintenance.  In  this  report  we  will  refer  to  the  choice  of  pub¬ 
lic  or  private  sector  as  the  depot  maintenance  repair  sector  selection  (RSS)  deci¬ 
sion. 

The  RSS  decision  occurs  in  two  contexts:  workload  associated  with  new  systems 
not  yet  in  the  inventory;  and  workload  associated  with  systems  already  in  the  in¬ 
ventory,  for  which  either  the  public  or  private  sector  might  be  the  incumbent  pro¬ 
vider.  The  issues  we  raise  and  the  broad  recommendations  we  make  are  intended 
to  be  generally  applicable  to  both  contexts.  However,  the  proposed  decision  model 
and  methodology  developed  as  part  of  the  study  were  use-tested  only  on  new  sys¬ 
tems. 

Background 

Depot  maintenance  entails  repair,  rebuilding,  and  major  overhaul  of  weapon  sys¬ 
tems  (e.g.,  ships,  tanks,  and  aircraft),  parts,  assemblies,  and  subassemblies.  DoD 
currently  expends  about  $12  billion  annually  for  depot  maintenance.  Depot  main¬ 
tenance  is  performed  by  both  the  public  (government-owned  and  -operated)  and 
private  sectors.  DoD  depots  perform  approximately  70  percent  of  the  work,  based 
on  dollar  value,  and  commercial  sources  30  percent. 

Numerous  changes  have  occurred  in  the  public  and  private  sectors  that  affect  this 
division  of  work.  First,  large-scale  reductions  in  the  cost  of  transportation  and  in¬ 
formation  relative  to  other  factors  such  as  inventory  and  labor  have  helped  fuel  a 
well-publicized  shift  to  outsourcing  in  the  commercial  sector.  Given  the  perceived 
benefits,  it  is  small  wonder  that  some  senior  DoD  executives  are  pushing  for 
greater  outsourcing  (i.e.,  privatization)  of  depot  maintenance.  Second,  in  the  face 
of  a  decline  in  the  acquisition  of  new  weapon  systems,  original  equipment  manu¬ 
facturers  (OEMs)  and  their  major  subcontractors  have  shown  a  much  stronger 
interest  in  this  work  than  has  historically  been  the  case.  Third,  third-party  mainte¬ 
nance  providers  have  emerged  with  technical  capabilities  generally  equivalent  to 
that  of  the  public  depots.  And  fourth,  the  total  amount  of  depot  maintenance  work 
has  decreased  since  the  end  of  the  Cold  War,  with  the  result  that  the  public  and 
private  sectors  are  vigorously  contending  for  slices  of  a  diminishing  pie. 
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Not  surprisingly,  given  the  above  background,  the  available  procedures  for  allo¬ 
cating  work  between  the  public  and  private  sectors  are  proving  less  than  satisfac¬ 
tory.  Both  internal  and  external  critics  continue  to  call  into  question  their 
adequacy,  especially  in  terms  of  comprehensiveness,  consistency,  fairness,  and 
effectiveness  in  mitigating  performance  risk.  To  some  extent  these  critical  con¬ 
cerns  are  legitimate;  to  some  extent  they  reflect  the  competing  sectors’  parochial 
interests  in  retaining  or  obtaining  workload.  Regardless,  the  department  does  not 
have  a  process  for  making  RSS  decisions  in  which  the  various  constituencies  can 
place  confidence. 

What  are  the  procedures  for  making  repair  sector  selection  decision  methods? 
DoD  has,  at  various  times,  used  decision  tree  analysis  (DTA),  life-cycle  cost 
(LCC)  analysis  and,  most  recently,  the  CORE  methodology  (i.e.,  a  specific 
method  used  to  decide  what  is  and  what  is  not  core  workload)  to  make  RSS  deci¬ 
sions.  The  DTA  method,  which  depended  on  simple  yes-no  responses  to  a  chain 
of  questions,  was  an  unsophisticated  approach  to  a  complex  problem  that  inher¬ 
ently  lacks  simple  answers.  LCC  analysis,  by  focusing  exclusively  on  cost,  ignores 
potentially  crucial  differences  in  relative  performance  risk.  And,  in  any  event, 
analysis  of  public  and  private  options  often  do  not  indicate  a  clear  cost  advantage 
for  either  sector.1  Although  the  CORE  methodology  addresses  risk  and  initially 
appeared  to  hold  promise,  in  practice  it  has  proved  difficult  to  decide  in  any  con¬ 
sistent  way  what  needs  to  be  in  core  workload  and  what  does  not.  For  instance,  all 
three  military  services  entrust  some  frontline  weapons  to  commercial  sources  of 
repair.  This  fact  of  life  suggests  that  performance  by  federal  government  employ¬ 
ees  in  organic  maintenance  depots  is  not  always  essential,  but  leaves  unclear  the 
circumstances  where  it  is  not  essential.  The  final  consideration  is  that,  in  general, 
all  of  the  methods  have  narrowly  focused  on  depot  maintenance  per  se  rather  than 
considering  depot  maintenance  as  one  component  of  a  supply  chain  extending 
from  factory  to  military  end  user. 

Study  Approach 

Our  approach  to  reexamining  the  RSS  decision  comprised  four  steps. 

Survey  of  Related  Research 

First,  we  surveyed  other  relevant  work  in  this  area.  Since  1993  there  have  been 
five  major  studies  of  depot  maintenance:  by  the  Joint  Chiefs  of  Staff,  the  Deputy 
Under  Secretary  of  Defense  (Logistics),  the  Commission  on  Roles  and  Missions 
of  the  Armed  Forces,  the  Defense  Science  Board  Task  Force  on  Depot  Mainte¬ 
nance  Management,  and  the  Congressional  Budget  Office.  There  have  also  been  a 
number  of  studies  by  the  Logistics  Management  Institute  (LMI),  the  RAND  Cor¬ 
poration,  the  Center  for  Naval  Analyses,  Coopers  and  Lybrand,  and  others.  Each 
of  these  studies  contributed  important  insights  but  collectively,  in  our  judgment, 
they  were  not  a  sufficient  basis  on  which  to  design  an  improved  RSS  decision 
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process.  To  fill  voids  we  conducted  additional  research  in  the  areas  of  classical 
microeconomics,  transaction  cost  economics,  public  choice  theory,  organizational 
theory,  technology  management,  and  supply  channel  behavior. 

Interviews  of  DoD  Personnel 

Second,  we  conducted  semi  structured  interviews  of  DoD  management  personnel 
in  acquisition  program  management,  depot  maintenance  management,  materiel 
management,  and  weapon  system  operations. 

Formulation  of  Decision  Model  and  Process 

Third,  we  synthesized  the  results  of  the  literature  search  and  the  interviews  to  cre¬ 
ate  a  new  RSS  decision  model  and  accompanying  decision  process. 

Decision  Model 

The  initial  result  of  the  literature  review  and  interviews  was  a  lengthy  list  of  can¬ 
didate  decision  elements  that  ranged  from  customer  needs  such  as  low  cost  to 
more  indirect  considerations  such  as  pipeline  size,  existence  of  more  than  one 
commercial  source,  and  relative  labor  rates.  We  did  not  find  a  ready-made  unify¬ 
ing  theme  for  the  various  elements.  However,  it  was  possible  to  accommodate  all 
elements  in  a  three- tiered  model  (Figure  1-1)  that  interrelates 

♦  customer  needs, 

♦  characteristics  of  alternative  sources  of  repair,  and 

♦  three  sets  of  determining  factors  (a  total  of  27  factors,  initially),  which 
either 

►  are  intrinsic  to  the  nature  of  the  repair  sources  themselves, 

►  depend  on  the  relationship  between  the  repair  source  and  the  buyer  of 
repair  services,  or 

>-  are  a  function  of  the  repair  work  to  be  accomplished. 

Because  there  are  a  significant  number  of  important  factors  and  none  lend  them¬ 
selves  to  simple  yes-no  answers,  evaluation  is  too  complex  for  a  decision  tree. 
Accordingly,  we  developed  an  arithmetic  approach  that  combines  individual  fac¬ 
tor  evaluations  into  figures  of  merit,  which  indicate  the  relative  attractiveness  (or 
risk)  of  the  organic  and  private  sectors. 
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Figure  1-1.  RSS  Decision  Model 
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Decision  Process 


The  decision  process,  illustrated  in  Figure  1-2,  was  based  on  an  integrated  process 
team  (IPT)  approach;  in  this  case  IPTs  comprised  both  government  and  industry 
representatives.  The  primary  motivation  for  this  design  was  to  incorporate  a 
mechanism  for  countering  the  inevitable  biases  that  result  when  individuals  with  a 
stake  in  an  outcome  are  asked  to  make  evaluations. 

The  basic  decision  process — evaluation  of  established  factors,  integration  in  a 
simple  mathematical  model,  interpreting  results,  followed  by  holistic  assess¬ 
ment — intentionally  borrows  from  successful  methods  used  for  contract  source 
selection.  The  mathematical  model  combines  a  number  of  disparate  and  often 
subjective  evaluations  into  a  much  smaller  set  of  figures  of  merit  that  are  rela¬ 
tively  easy  to  interpret.  However,  precise  appearing  numerical  scores  can  also  give 
the  results  more  credence  than  they  deserve — hence  the  holistic  assessment  is  an 
essential  part  of  the  decision  process. 
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Figure  1-2.  RSS  Decision  Process 


1 .  Joint  [SPO,  depot,  and 
industry]  evaluation  of 
factors  in  three  areas 


2.  RSS  decision  model 


3.  Interpreting  results 


Relative  risk 

•Overall  comparison 

High 

AKA  AltB  Low 

*  Y 

.  YY 

•Agility/responsivenes 

Y  Y 

■RopnoMIO 

T . V . 

•Quality  product  &  service 

•Low  total  cost 

Y  Y 

•Continuous  improvement 

_ 

I"1 

jC 

Program  manager  holistic 
evaluation,  RSS  decision 


Test  of  the  Model  and  Process 

As  the  fourth  step,  we  use-tested  the  decision  model  and  process  on  three  different 
V-22  aircraft  workloads.  In  order  to  use-test  the  decision  model: 

♦  We  implemented  the  model  in  a  spreadsheet. 

♦  Next,  the  importance  of  each  customer  need  was  initially  assigned  by  LMI 
analysts  and  then  verified  as  reasonable  during  field  interviews.  LMI, 
using  a  focus-group,  also  developed  a  preliminary  set  of  weights  linking 
customer  needs  to  the  characteristics  of  repair  sector  choices  and  another 
set  of  weights  linking  characteristics  of  choices  to  determining  factors. 

♦  Using  the  decision  model  with  LMI-developed  weights,  the  V-22  system 
program  office  (SPO)  convened  IPTs  to  conduct  RSS  evaluations  on  the 
T-406  engine,  the  V-22  forward-looking  infrared  scanner,  and  the  V-22 
Cockpit  Control  Feel  and  Drive  actuator. 


Results 


In  general,  IPT  participants  were  comfortable  with  the  proposed  RSS  decision 
model  and  saw  it  as  providing  value-added  structure  to  the  RSS  decision  process. 
In  fact,  the  process  helped  the  V-22  SPO  reach  and  defend  a  decision  on  the 
T-406  engine.  However,  the  results  of  the  use-tests  supported  only  22  of  the 
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proposed  factors  (Table  1-1).  We  recommend  these  22  factors  be  incorporated 
into  the  policy  on  RSS  decisions. 


Table  1-1.  RSS  Decision  Factors 


Category 

Decision  factor 

Nature  of  the  repair  work 

• 

Relative  size  versus  potential  source  of 
repair 

• 

Demand  predictability 

Design  stability 

• 

Repair  process  dependence 

• 

Absence  of  proprietary  data  issues 

Nature  of  the  relationship  with  potential 

• 

Flexibility  of  relationship 

sources  of  repair 

• 

Scope  of  relationship 

• 

Policy  on  related  functions  such  as  en¬ 
gineering  and  materiel  management 

• 

Policy  on  lot  sizes 

• 

Policy  on  repair  parts  management 

• 

Incentive  structure 

Nature  of  the  sources  of  repair 

• 

Availability  of  more  than  one  source 

• 

Availability  of  sources  with  comple¬ 
mentary  workload 

• 

Interest  by  potential  sources 

• 

Reserve  capacity  for  this  work 

• 

Capacity  in  excess  of  that  needed  for 
workload 

• 

Availability  of  skilled  labor 

• 

Stable  labor  relationships 

• 

Customer  knowledge 

• 

Product  knowledge 

• 

Process  knowledge 

• 

Implementation  of  closed-loop  Manu¬ 
facturing  Resource  Planning  (MRP-II)- 
based  management  methods 

The  use-tests  indicate  that  the  work  required  for  an  IPT  to  subjectively  evaluate 
22  factors  is  not  exorbitant  using  the  model  we  developed.  An  IPT  meeting  of  4  to 
5  hours  was  sufficient  even  for  complex  workloads,  given  adequate  but  not  unrea¬ 
sonable  preparation. 

The  concept  of  capacity,  however,  will  require  further  research.  The  two-part  con¬ 
struct  we  used  (reserve  and  excess  capacity)  presented  difficulty  in  all  tests.  It  is 
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possible  that  measuring  capacity  may  require  more  effort  than  the  benefits  war¬ 
rant.  If  so,  reserve  and  excess  capacity  could  be  dropped  from  the  decision  model. 

Of  equal,  if  not  greater,  importance  is  that  the  IPT  format  did  not  fully  control 
bias.  Although  gathering  both  sides  to  an  RSS  decision  at  the  same  table  has  its 
merits,  the  tests  revealed  some  unresolved  problems: 

♦  The  potential  for  one  or  more  strong  individuals,  particularly  those  intent 
on  introducing  bias,  to  dominate  the  session. 

♦  The  potential  for  opposing  representatives  to  not  be  peers  from  the  stand¬ 
point  of  education,  technical  training,  or  experience.  As  an  example,  there 
is  a  potential  that  industry  representatives  will  be  more  sophisticated  and 
better  prepared  than  their  government  counterparts. 

♦  A  lack  of  relevant  information.  When  people  know  which  depot  or  com¬ 
mercial  source  of  repair  would  likely  receive  the  work,  finding  individuals 
who  can  knowledgeably  assist  in  the  factor  evaluations  is  reasonably 
straightforward.  This  is  not  the  case  if  the  organic  and  commercial  sources 
are  up  in  the  air. 

♦  RSS  evaluations  are  typically  made  by  the  staffs  of  SPOs.  These  individu¬ 
als  may  lack  prior  experience  in  such  decisions  or  any  related  training. 
Further,  once  they  make  the  investment  in  mastering  the  process  they  may 
not  have  the  opportunity  to  participate  in  such  decisions  again. 

To  identify  alternatives  to  an  IPT,  we  looked  for  promising  analogues  in  other 
spheres.  Finding  such  an  analogue  was  not  difficult.  The  central  problems  of  the 
RSS  decision — providing  for  experienced  and  trained  staff,  uncovering  all  of  the 
relevant  information,  assuring  freedom  from  conflict  of  interest — occur  also  in  the 
context  of  major  business  transactions  such  as  mergers  and  public  offerings. 

There,  these  problems  are  addressed  through  what  is  known  as  a  “due  diligence” 
procedure.  The  basics,  if  not  necessarily  the  details,  of  due  diligence  are  applica¬ 
ble  to  the  RSS  decision:  due  diligence  succeeds  when  it  is  performed  by  investi¬ 
gators  who  have  the  requisite  training  and  experience  and  are  also  free  from 
conflicts  of  interest.  Accordingly,  we  recommend  that  DoD  borrow  from  the  due 
diligence  practice  and  create  an  RSS  decision  support  cadre.  This  cadre  could  no- 
tionally  report  to  the  Defense  Depot  Maintenance  Council.  Further,  it  should  be 
constituted  such  that  it  is 

♦  truly  impartial  regarding  the  outcome  of  RSS  decisions  and  able  to  main¬ 
tain  the  confidence  of  the  competing  constituencies,  as  well  as  critics  of 
the  process  or  decision; 

♦  able  to  bring  to  the  RSS  analyses  the  requisite  education,  technical  train¬ 
ing,  market  awareness,  and  policy  perspectives — in  particular,  cadre  staff 
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should  be  sufficiently  senior  to  be  on  a  peer  level  with  their  organic  depot 
and  commercial  industry  counterparts;  and 

♦  an  aid  to  the  services  in  performing  RSS  decisions,  but  does  not  take  the 
decisions  out  of  their  hands. 

Limitations 

The  evaluation  approach  described  in  this  report  focuses  on  the  relatively  immedi¬ 
ate  requirements  of  the  repair  customer  and  what  it  takes  to  satisfy  those  require¬ 
ments.  This  approach  excludes  a  number  of  potentially  important  considerations. 
Among  them  are  the  political  considerations  mentioned  above,  as  well  as  equity 
considerations  (for  example,  effects  on  the  government  or  contract  work  force  of 
shifting  workload  from  one  to  the  other).  Additionally,  because  the  approach  is 
inherently  incremental,  workload-by-workload,  it  does  not  address  issues  relating 
to  long-term  strategic  posture.  Finally,  as  mentioned  earlier,  the  proposed  ap¬ 
proach  has  been  tested  only  on  new  workloads. 
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End  Notes  for  Chapter  1 


1  Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Report  of  the 
Defense  Science  Board  Task  Force  on  Depot  Maintenance  Management ,  April  1994,  p.  F-38; 

David  R.  Warren,  Uncertainties  and  Challenges  DoD  Faces  in  Restructuring  Its  Depot  Mainte¬ 
nance  Program ,  Statement  before  the  Subcommittee  on  Military  Readiness,  Committee  on 
National  Security,  U.S.  House  of  Representatives,  18  March  1997,  p.  25. 
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Chapter  2 

Designing  a  New  Decision  Model 

Overview 


As  described  in  Chapter  1,  DoD  has,  at  various  times,  used  methods  such  as  deci¬ 
sion  tree  analysis  (DTA),  life-cycle  cost  (LCC)  analysis,  and,  most  recently,  the 
CORE  methodology  to  make  RSS  decisions.  None  has  proved  satisfactory.  DTA, 
which  depends  on  yes-no  responses  to  a  series  of  questions,  was  an  overly  simpli¬ 
fied  approach  to  a  complex  problem.  LCC  analysis,  by  focusing  exclusively  on 
cost,  ignores  potentially  crucial  differences  in  relative  performance  risk.  Although 
the  CORE  methodology  was  more  encompassing  and  held  promise,  in  practice  it 
has  proved  difficult  to  decide  in  any  consistent  way  what  needs  to  be  in  CORE 
and  what  does  not. 

Developing  a  new  approach  to  the  depot  RSS  decision  requires  two  considera¬ 
tions.  The  first  is  a  reasonably  comprehensive  statement — i.e.,  a  model — of  the 
key  elements  of  the  RSS  decision  itself.  The  second  is  a  means  for  applying  the 
decision  model  fairly  and  consistently  across  the  military  services  and  over  time. 
This  chapter  develops  the  structure  for  and  candidate  factors  to  be  included  in  the 
new  model;  Chapter  3  then  develops  a  process  for  applying  the  model.  The  model 
and  process  are  refined  in  Chapter  4  based  on  the  results  of  trial  applications. 

Review  of  DoD  experience  to  date,  related  trends  in  commercial  industry,  and  the 
research  literature  resulted  in  a  lengthy  and  heterogeneous  list  of  candidate  issues 
and  factors  that  could  bear  on  the  RSS  decision.  Although  we  did  not  find  a  ready¬ 
made  unifying  structure  for  these  various  factors  (actually,  there  were  multiple 
and  conflictual  unifying  structures),  we  were  able  to  accommodate  them  in  a  fairly 
simple  model  with  three  basic  elements  (Figure  1-1).  The  three  elements  are: 

♦  Needs  of  the  end  users  of  depot  repair  services  (e.g.,  timeliness,  low  cost, 
quality) 

♦  Characteristics  of  the  two  repair  sectors  relevant  to  satisfying  user  needs 
(e.g.,  adequate  capability  and  capacity,  scope  and  scale  economy,  respon¬ 
siveness) 

♦  Factors  that  determine  the  characteristics  of  the  repair  sectors.  These  fac¬ 
tors 

►  are  intrinsic  to  the  nature  of  the  repair  sources  themselves, 
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►  depend  on  the  relationship  between  the  repair  source  and  the  buyer  of 
repair  services,  or 

>-  are  a  function  of  the  repair  work  to  be  accomplished. 

The  balance  of  this  chapter  develops  each  of  these  elements. 

Needs  of  End  Users 

The  basic  and  well-documented  needs  of  the  end  users  of  depot  repair  services  are 
to  minimize  operational  risk  (through  assured  readiness  and  sustainability)  and 
control  costs.  However,  readiness  and  sustainability  are  relatively  broad  concepts, 
and  it  is  not  necessarily  clear  what  is  meant  by  the  term  “cost.”  Hence,  it  is  neces¬ 
sary  to  break  these  broad  requirements  down  into  more  specific  subrequirements. 
The  body  of  literature  on  depot  maintenance  suggests  five  basic  depot  mainte¬ 
nance  customer  requirements  that,  if  satisfied,  minimize  risk  and  control  cost: 

♦  Ability  to  meet  basic  demand 

♦  Agility  and  responsiveness  to  changing  demand 

♦  Quality  product  and  service 

♦  Low  total  cost 

♦  Continuous  improvement  of  both  product  and  service. 

Below  we  describe  each  of  these  requirements  and  show,  by  reference  to  policy 
and  previous  studies,  why  they  are  legitimate  depot  maintenance  requirements. 

Ability  to  Meet  Basic  Demand 

The  ability  to  meet  basic  demand  means  remanufacturing  reparables  at  the  average 
or  steady-state  rate  expected  in  peacetime.  It  supports  the  requirement  to  keep 
weapon  systems  in  a  high  state  of  operational  readiness  during  peacetime — i.e.,  to 
support  peacetime  training  and  normal  peacetime  operational  tasks,  such  as  alert 
and  airlift/sealift,  at  their  planned  operational  tempos.  However,  we  use  the  nar¬ 
rower  construct  of  meeting  demand  (for  remanufacture  of  reparables)  rather  than 
readiness,  because  there  are  many  influences  on  readiness,  of  which  ability  to  re¬ 
manufacture  reparables  at  an  adequate  rate  is  only  one. 
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Agility/Responsiveness 

Agility/responsiveness  is  the  ability  to  satisfy  an  unanticipated  change  in  require¬ 
ments.  There  are  three  types  of  changes  in  requirements: 

♦  Surge 

♦  Changes  in  priorities 

♦  Qualitatively  different  requirements. 

Surge 


Probably  the  most  familiar  responsiveness-related  requirement  is  an  overall  in¬ 
crease  in  demand — generally  referred  to  as  surge.  Surge  was  an  assumed  require¬ 
ment  during  the  Cold  War.  A  case  can  be  made  that,  while  surge  was  an  important 
consideration  for  a  protracted  conflict,  it  may  not  be  during  a  contingency.1  How¬ 
ever,  because  there  is  not  yet  a  consensus  on  this  issue  and  because  there  was  evi¬ 
dence  of  surge  during  the  Persian  Gulf  conflict  in  all  three  services,2  we  retain  it 
as  a  customer  requirement. 

Changes  in  Priorities 

Of  potentially  more  critical  importance  are  changes  in  priorities,  particularly  for 
high-technology  weapon  system  components  such  as  electronics  and  turbine  en¬ 
gines.  A  growing  body  of  research  indicates  that  constant  changes  in  depot  main¬ 
tenance  priorities  should  be  considered  the  norm  rather  than  the  exception  and  that 
there  is  a  significant  penalty  if  a  source  of  repair  cannot  respond  to  them.3  One 
study4  summarized  the  situation  well  when  it  concluded  that 

♦  the  variability  of  resource  demands  thwarts  accurate  forecasting  even  in 
peacetime; 

♦  wartime  forecasting  is  even  more  difficult  because,  in  addition  to  the  in¬ 
herent  problem  of  demand  variability  (which  gets  worse  in  wartime),  it  is 
unlikely  that  the  real  contingency  will  ever  match  the  planning  assump¬ 
tions  that  were  made;  and 

♦  during  a  contingency,  there  are  unpredictable  threats  to  repair,  supply,  and 
transportation. 

Qualitatively  Different  Requirements 

Unanticipated  increases  and  decreases  in  steady-state  demand  rates  are  not  the 
whole  story;  qualitative  changes  in  requirements  are  also  important.  An  example 
would  be  the  rapid  dispatch  of  field  teams  for  in-theater  and  crash  or  battle  dam¬ 
age  repair. 
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Quality  Product  and  Service 

A  third  customer  requirement  is  quality.5  Quality  can  take  on  a  range  of  meanings, 
from  the  ability  to  satisfy  customer  needs  in  the  broadest  sense  to  the  narrower 
sense  of  compliance  with  performance  or  material  specifications.6  In  this  context 
the  broader  meaning  would  be  potentially  confusing,  since  it  would  include  such 
considerations  as  agility  and  responsiveness.  To  avoid  this  potential  confusion,  we 
will  define  quality  to  include  product  quality  (performance  and  materiel  specifica¬ 
tions)  and  service  quality. 

Low  Total  Cost 

Virtually  all  of  the  literature  on  depot  maintenance  and  almost  all  interviewees 
cited  cost  as  an  important  consideration  in  making  source  of  repair  decisions. 

Generally,  however,  and  as  Camm  has  noted,  cost  is  narrowly  defined  when  used 
as  a  criterion  for  the  source  of  repair.7  Typically  it  encompasses  only  the  cost  of 
repair  (the  sum  of  direct  labor,  depreciation  on  capital  plant,  and  other  depot  indi¬ 
rect  costs).8  Absent  from  the  usual  definition  is  the  investment  in  reparable  in¬ 
ventory  to  fill  the  pipeline  to  and  from  the  source  of  repair.  This  cost  is  significant 
and  is  affected  by  the  characteristics  of  the  source  of  repair.9  For  this  reason  we 
explicitly  include  pipeline  costs. 

There  is  another,  more  subtle,  type  of  cost  that  is  also  generally  assumed  away — 
the  cost  of  administration,  by  which  we  mean  the  costs  of  writing  an  initial  con¬ 
tract,  negotiating  changes  in  performance  to  contracts  that  already  exist,  and 
monitoring  performance.10  (In  this  case  the  word  “contract”  is  a  generic  term  that 
applies  to  both  legal  contracts  with  commercial  sources  of  repair  and  to  the  more 
informal  arrangements  with  DoD  maintenance  depots.)  We  noted  above  in  the 
discussion  on  agility  and  responsiveness  that  it  is  impossible  to  forecast  demands 
with  exact  accuracy,  particularly  in  a  contingency.  In  the  case  of  DoD  organic 
repair  (assuming  effective  organizational  arrangements),  such  an  inability  to  fore¬ 
cast  might  not  matter  much  because,  as  with  any  internal  command  organization, 
it  is  at  least  nominally  possible  to  rapidly  adjust  output  as  changes  in  requirements 
become  known.  It  is  generally  harder,  and  potentially  much  harder,  to  provide  for 
such  contingent  performance  through  a  formal  contract  mechanism. 

Need  for  rapid  adjustment  in  output  (contingent  performance)  is  the  fundamental 
rationale  for  the  DoD  CORE  policy  and  for  viewing  DoD  internal  CORE  depot 
capability  as  a  “ready  and  controlled”  source.  Although  not  usually  recognized  as 
being  so,  the  CORE  policy  is  well  grounded  in  economic  transaction  cost  theory11 
and  the  literature  on  supply  chain  management.12  Commercial  firms  adopt  similar 
policies  to  guide  their  make-or-buy  decisions. 

However,  that  is  not  to  say  it  is  impossible  to  provide  for  contingent  performance 
from  a  commercial  source  of  repair,  or  to  protect  against  potential  price  gouging 
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when  negotiating  follow-on  contracts  when  there  is  a  single  seller.  These  things 
can  be  provided  for,  but  the  administrative  cost  of  doing  so  is  normally  larger  than 
for  using  an  organic  source  of  repair. 

Given  the  above  considerations,  we  define  total  cost  to  include  the  following: 

♦  The  cost  of  maintenance,  per  se,  including  depot  noncapital  indirect  costs 

♦  Capital  costs  of  facility  and  pipeline.  As  capital  costs,  these  costs  are  gen¬ 
erally  one-time,  start-up  costs. 

♦  The  cost  of  administration. 

Continuous  Improvement  of  Product  and  Service 

The  fifth,  and  last,  requirement  is  continuous  improvement  of  product  and  service. 
It  is  generally  absent  from  previous  studies  of  depot  maintenance.  However,  it  is 
not  a  novel  concept  and  has  received  much  emphasis  in  the  manufacturing  litera¬ 
ture  during  the  last  decade  as  a  legitimate  user  requirement.  We  include  it  here 
because  of  its  potential,  if  not  fully  recognized,  importance  to  DoD  depot  mainte¬ 
nance. 

Summary  of  Customer  Requirements 

In  summary,  then,  these  are  the  five  customer  requirements: 

♦  Meeting  basic  demand. 

♦  Agility/responsiveness  in  providing  for 
>-  reprioritization, 

►  surge,  and 

►  new  and  qualitatively  different  requirements. 

♦  Quality  product  and  service. 

♦  Low  total  cost  of 

►  maintenance,  per  se, 

>-  administration,  and 

►  start-up  capitalization. 

♦  Continuous  improvement  of  product  and  service. 
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Characteristics  of  the  Repair  Sectors 


The  section  above  identified  five  basic  user  requirements.  How  well  they  are  satis¬ 
fied  will  depend  on  the  characteristics  of  the  chosen  source  of  repair.  Before  pro¬ 
ceeding  with  a  discussion  of  the  characteristics  of  the  choices,  it  is  worthwhile  to 
recall  that  in  the  context  of  this  study  the  choice  is  not  between  a  specific  firm  and 
specific  government  depot,  but  the  more  general  choice  between  organic  and 
commercial  repair  sources — where  any  of  a  number  of  commercial  firms  or  gov¬ 
ernment  depots  could  be  the  actual  future  provider. 

In  this  section  we  develop  10  repair  sector  characteristics  that  could  determine 
how  well  customer  requirements  are  met.  It  is  certainly  possible  to  generate  a  list 
with  more  or  less  than  10  characteristics.  The  list  here  results  from  balancing  three 
goals: 

♦  Creating  a  list  short  enough  to  be  of  practical  use 

♦  Being  specific 

♦  Keeping  the  characteristics  from  overlapping. 

The  10  characteristics  are  shown  in  summary  form  in  Figure  2-1.  In  the  text  that 
follows  we  define  the  characteristics  and  show,  primarily  by  citing  customer 
requirements,  why  they  are  relevant  to  the  source  of  repair  decision. 

Figure  2-1.  Characteristics  of  Repair  Sector  Choices 


•  Adequate  capability  and  capacity  • 

•  Scope  and  scale  economy  • 

•  Overhead  costs  • 

•  Non-value-added  effort 

•  Extraordinary  profits,  fees,  earnings 

•  Responsiveness  to  changing  needs 

-  Reprioritization  capability 

-  Surge  capability  • 

-  Task  flexibility 

-  Customer  linkage 

-  Administrative  ease 


Incentive  to  improve 
Work  stoppage  protection 
Pipeline  size  (quantity  and  time) 

-  Order  and  ship  (administration) 

-  Repair  cycle  at  source  of  repair 

-  Material  movement  (user  to  SOR 
and  return) 

Output  quality 

-  Of  product 

-  Of  service 
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Adequate  Capability  and  Capacity 

Two  of  the  customer  needs  were  ability  to  meet  basic  demand  and  the  agility  and 
responsiveness  to  satisfy  unanticipated  changes  in  demand.  In  part  these  are  capa¬ 
bility  and  capacity  issues  where,  consistent  with  the  theory  of  constraints,  capa¬ 
bility  and  capacity  are  both  zero  until  each  significant  production  constraint  has 
been  removed.  At  that  point  there  is  capability  and  some  (not  necessarily 
enough)  capacity. 

To  increase  capacity,  more  resources  have  to  be  provided  at  the  points  that  con¬ 
strain  the  production  process.  Hence  adequate  capability  and  adequate  capacity 
entail  having  in  place  both  the  technical  processes  necessary  to  repair  what  is 
needed  (capability)  and  also  the  resources  to  assure  production  at  required  rates 
(capacity).  Capacity  is  important  to  both  the  ability  to  produce  when  needed  and 
to  cost.  If  the  volume  of  goods  produced  falls  below  capacity,  then  costs  per  unit 
rise  rapidly,  assuming  no  alternative  use  of  resources. 

Scale  and  Scope  Economy 

We  established  earlier  that  control  of  costs  is  an  important  user  need.  Two  im¬ 
portant  characteristics  of  the  components  of  military  weapon  systems  are  their  low 
individual  demand  (failure)  rates14  and,  as  we  will  also  discuss  later,  the  signifi¬ 
cant  variability  of  those  rates.15  Low  repair  volume  per  reparable  item  (i.e.,  part- 
numbered  or  stock-numbered  item)  makes  it  inherently  difficult  to  operate  effi¬ 
ciently  if  repairs  are  performed  at  many  individually  small  locations.  DoD  depots 
have  sought  efficiency  by  centralizing  repairs  for  individual  reparable  items  at  one 
or  at  least  a  small  number  of  locations  and  by  establishing  facilities  that  can  repair 
a  broad  range  of  complementary  items.  The  first  approach  is  generally  described 
as  scale  economy  and  the  second  as  scope  economy.  In  1985,  Embry  et  al.  held 
that  no  commercial  aviation  sources  of  repair  had  broad  scope,16  but  the  same  is 
not  necessarily  true  today.17  The  major  commercial  shipyards,  of  course,  have  tra¬ 
ditionally  had  a  broad  scope  of  repair  capabilities. 

Overhead  Costs 

Direct  costs  accrue  from  the  unit  being  produced.18  The  three  major  components 
are  direct  labor,  direct  materials,  and  purchased  parts.  Overhead  (or  indirect)  costs 
are  all  other  costs  incurred  in  accomplishing  repairs  and  running  the  business. 
Examples  are  management  salaries,  utilities,  equipment  depreciation,  and  training. 
Because  control  of  overhead  is  a  significant  issue,  it  has  figured  prominently  in 
just  about  every  recent  look  at  depot  maintenance.  What  makes  it  particularly  per¬ 
tinent  to  the  repair  sector  selection  decision  is  that  the  overhead  rates  of  various 
potential  providers  of  depot  maintenance  are  believed  to  differ  significantly. 
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Non- Value- Added  Effort 


The  distinction  between  direct  and  indirect  costs  comes  from  accounting.  There  is 
a  second  and  more  recent  accounting  distinction  between  value-added  and  non- 
value-added  effort.  Value-added  effort  “is  the  contribution  made  by  an  operation 
or  a  plant  to  the  final  usefulness  and  value  of  a  product  as  seen  by  the  customer.”19 
Non-value-added  effort,  which  may  be  either  direct  or  indirect  in  the  traditional 
sense,  is  effort  that  adds  to  cost  but  does  not  contribute  to  value  as  seen  by  the 
customer.  Obvious  examples  for  depot  maintenance  would  be  replacing  parts  not 
subject  to  wearing  out,  inspecting  and  testing  parts  (such  as  electronics)  where 
incipient  failure  cannot  be  predicted,  and  repairing  features  that  the  user  does  not 
want  or  use.  Non- value- added  effort  patently  exists  in  depot  maintenance  as  it 
does  in  any  manufacturing  enterprise,  particularly  in  the  form  of  “nice-to-have” 
services.  Because  the  extent  of  unnecessarily  added  cost  may  depend  on  the  repair 
sector  selected,  we  include  non-value-added  effort  as  a  repair  sector  characteristic. 

Extraordinary  Profits,  Fees,  Earnings 

When  a  product  or  process  entails  significant  sunk  costs  that  are  specific  to  either 
the  user’s  or  producer’s  operation,  the  party  with  the  least  sunk  costs  has  an  op¬ 
portunity  to  take  advantage  of  the  other.20  DoD  weapon  systems  for  which  there 
are  no  commercial  equivalents  are  such  sunk  costs  and  might  offer  such  an  op¬ 
portunity,  when  there  is  scant  choice  for  repair  capability  in  the  commercial  sector 
other  than  by  the  OEM.  Hence  there  is  a  legitimate  basis  for  fearing  exploitation 
by  opportunistic  suppliers.21  (This  issue  is  referred  to  in  the  transaction  cost  lit¬ 
erature  as  the  asset  specificity  problem — recognizing  the  situation  where  assets 
are  specific  to  a  particular  use  and  not  easily  used  for  some  alternative  purpose.) 

As  an  example,  if  there  were  only  one  commercial  source  of  depot  maintenance 
for  B-2  aircraft,  then  that  commercial  source  would,  at  least  notionally,  have  the 
leverage  to  extract  larger  payments  for  repair  services  than  if  there  were  more 
sources.  (This  problem  is  actually  a  two-edged  sword:  if  a  commercial  firm  in¬ 
vests  in  a  repair  capability  for  which  the  only  customer  is  DoD,  then  DoD  gains 
leverage.  Here  we  are  primarily  concerned  with  situations  where  the  government 
is  at  risk.)  It  is  not  a  given,  however,  that  suppliers  will  act  opportunistically  when 
offered  the  chance,  and  each  situation  merits  individual  consideration.  For  all  of 
these  reasons,  extraordinary  profits,  fees,  and  earnings  are  considered  as  a  char¬ 
acteristic  to  be  evaluated. 
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Responsiveness  to  Changing  Needs 

Consistent  with  the  user’s  needs  for  responsiveness,  there  are  five  components  to 
a  supplier’s  actual  responsiveness: 

♦  Reprioritization  capability 

♦  Surge  capability 

♦  Task  flexibility 

♦  Customer  linkage 

♦  Administrative  ease. 

Reprioritization,  surge,  and  task  flexibility  correspond  to  the  customer  needs  dis¬ 
cussed  earlier.  Customer  linkage  and  administrative  ease  are  further  explained 
below. 

Customer  Linkage 

There  is  obvious  credence  to  the  notion  that  warfighters  and  the  providers  of 
depot  maintenance  need  to  be  closely  linked.  In  fact,  the  1993  Office  of  the  Sec¬ 
retary  of  Defense  evaluation  of  depot  maintenance  management  options  used  as  a 
criterion  the  effect  on  the  linkage  between  the  warfighter  and  depot  mainte¬ 
nance.22  We  specifically  include  that  linkage  as  an  attribute  of  repair  sectors 
because  opponents  of  logistics  outsourcing  caution  that  outsourcing  will  lessen 
direct  contact  with  customers. 

Administrative  Ease 

In  the  information  processing  view  of  interorganizational  relationships,  for  pro¬ 
duction  to  be  effective  the  information  processing  capabilities  of  a  production 
structure  (one  organization  or  a  combination  of  organizations)  has  to  match  its 
information  processing  needs.  In  general,  the  required  capabilities  are  more  com¬ 
plex,  and  resulting  costs  are  larger,  for  contractual  arrangements  between  organi¬ 
zations  than  for  administrative  arrangements  within  an  organization.23  This  is 
what  the  Office  of  the  Secretary  of  Defense  (Logistics)  was  getting  at  in  its  inte¬ 
grated  management  study,  when  it  said  that  “public  sector  depots,  under  the  direct 
control  of  the  military,  are  organized  to  respond  rapidly  to  the  full  scope  of  main¬ 
tenance  requirements  to  be  expected  in  a  war”  (emphasis  added).24 

The  more  complex  capabilities  needed  to  administer  production,  if  it  is  done 
commercially  rather  than  organically,  normally  cost  more  and  are  more  difficult  to 
put  in  place.  Hence  our  coining  the  term  administrative  ease  and  including  it  as  a 
characteristic  of  the  repair  sector.  Note  that  administrative  ease  and  incentive  to 
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improve  (below)  can  pull  in  opposite  directions:  competition-based  incentives 
come  at  the  cost  of  increased  administrative  difficulty. 

Incentive  to  Improve 

We  noted  earlier  that  continuous  improvement  is  an  important,  if  under¬ 
recognized,  customer  requirement.  Since  improvement  does  not  just  happen  but  is 
caused,  understanding  the  incentives  to  improve  can  be  important,  if  different 
sources  of  repair  have  different  incentives. 

There  can  be  such  differences.  In  particular,  when  the  possibility  of  “exit” — 
choosing  another  supplier — is  real,  then  management  is  more  likely  to  take  seri¬ 
ously  the  needs  of  customers  than  when  there  is  no  alternative  supplier.  Such  dif¬ 
ferences  in  incentives  could  show  up  when  comparing  a  sole-source  public  depot 
with  competed  contract  support,  or  if  the  choice  is  between  sole-source  and  com¬ 
petitive  commercial  support.  Competition  can  create  strong  incentives  to  simplify 
organizations,  use  multiskilled  workers,  and  eliminate  unnecessary  work  in  order 
to  win  work.  Organization  size  also  affects  incentives,  with  incentives  generally 
being  impaired  in  larger  organizations.25 

Because  incentive  to  improve  can  differ  between  a  source  of  repair  that  has  com¬ 
petition  and  one  that  does  not,  or  between  a  large  organization  and  a  smaller  one 
(and  probably  for  other  reasons  as  well),  incentive  to  improve  is  a  relevant  char¬ 
acteristic  for  the  repair  sector  selection  decision. 

Work  Stoppage  Protection 

Protection  from  work  stoppage  or  disruption  is  one  of  the  arguments  for  preserv¬ 
ing  the  depot  maintenance  CORE  capability.  It  includes  protection  from  both  the 
effects  of  strikes  and  high  switching  costs,  the  costs  associated  with  switching 
from  one  supplier  to  another  in  the  event  the  first  supplier  is  not  satisfactory.26 
These  costs  need  not  be  economic  but  can  include,  for  instance,  an  impeded  abil¬ 
ity  to  effectively  wage  war.  The  implicit  assumption  in  the  CORE  policy  is  that 
organic  facilities  can  provide  this  protection  more  readily  than  can  commercial 
ones.  It  makes  sense,  then,  to  explicitly  include  this  as  a  feature  of  alternative  re¬ 
pair  sectors. 

Pipeline  Size  (Quantity  and  Time) 

Under  our  discussion  of  user  needs,  we  noted  that  low  total  cost  properly  includes 
investment  in  pipeline.  Pipeline  has  been  defined  as  the  distance  between  supplier 
and  consumer,  measured  in  days  of  supply;  thus  the  concept  of  pipeline  involves 
dimensions  of  both  quantity  and  time.27  With  regard  to  repair  sector  decisions, 
required  investment  in  pipeline  is  a  function  of  the  quantity  of  reparables  required 
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to  be  in  the  pipeline.  Here  we  use  the  term  “pipeline”  in  a  general  sense  to  encom¬ 
pass  all  reparables,  whether  or  not  they  are  in  transit. 

Defined  this  way,  the  pipeline  comprises  two  segments.  One  is  a  forward,  to-the- 
customer,  order-fulfillment  segment  (measured  in  terms  of  order  and  ship  time).28 
The  second  segment  is  usually  called  the  repair  cycle  (it  is  actually  a  half-cycle).  It 
includes  retrograde  movement,  repair-related  administration,  and  the  repair  proc¬ 
ess  itself. 29  In  order  to  better  align  the  pipeline  segments  with  the  concept  of 
administrative  ease  and  its  related  transaction  costs,  we  will  redefine  the  segments 
in  terms  of  activity  rather  than  sequence,  so  that  they  comprise 

♦  in-transit  movement  (both  user  to  repair  location  and  return); 

♦  administration— the  “order”  part  of  the  classical  order  and  ship  segment, 
as  well  as  administration  associated  with  repair  process  induction;  and 

♦  the  repair  process  at  the  source  of  repair  (depot  or  contractor  facility). 

Output  Quality 

Earlier,  we  established  the  customer  requirement  for  quality  and  defined  the  con¬ 
cept  to  include  both  product  and  service.  This  implies  fairly  obvious  output  meas¬ 
ures  for  the  source  of  repair.  Hence  we  include,  as  attributes  of  the  repair  sector, 
output  quality 

♦  of  product  and 

♦  of  service. 

Factors  that  Determine  Repair 
Sector  Characteristics 

At  the  start  of  this  chapter  we  identified  five  needs  of  the  end  users  of  depot  repair 
services:  the  ability  to  meet  basic  demand;  agility/responsiveness;  quality  product 
and  service;  low  total  cost;  and  continuous  improvement.  We  then  discussed  the 
characteristics  that  determine  how  fully  a  repair  sector  can  meet  customer  re¬ 
quirements. 

The  characteristics  of  the  repair  sectors,  in  turn,  are  determined  by  another  set  of 
factors  that  we  now  will  describe.  We  will  argue  that  there  are  three  evident 
classes  of  such  factors: 

♦  Factors  related  to  the  nature  of  work  itself.  Although  these  factors  are  a 
function  of  the  work  to  be  performed  rather  than  the  source  of  repair,  they 
may  make  one  repair  sector  more  attractive  than  another. 
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♦  Factors  describing  the  relationship  with  the  sources  of  repair  within  a  re¬ 
pair  sector.  This  second  category  of  factors,  which  includes  considerations 
such  as  incentive  structure,  is  primarily  a  function  of  policy.  This  category 
may  also  make  one  repair  sector  more  attractive  than  another  if,  for  in¬ 
stance,  the  policy  regarding  one  repair  sector  differs  from  the  policy  that 
affects  another. 

♦  Factors  related  to  the  sources  of  repair  themselves. 

Figure  2-2  summarizes  the  three  classes  of  factors. 


Figure  2-2.  Determining  Factors 


•  Nature  of  repair  work  itself 

Workload  size  vs.  size  of  market 
Workload  size  compared  to  the  workload 
at  sources  of  repair 
Demand  predictability 
Design  stability 
Technology  renewal  rate 
Uniqueness  of  technology 
Repair  process  dependence 
Availability  of  engineering  data 
Absence  of  proprietary  data  issues 

•  Nature  of  relationship  with  sources 

Type  of  relationship  (flexible  vs.  inflexible) 

Scope  of  relationship 

Policy  on  related  functions 

Policy  on  lot  sizes 

Policy  on  repair  parts 

Incentive  structure 


«  Nature  of  sources 

Availability  of  more  than  one  source 
Availability  of  sources  with  complementary 
workload 

Interest  by  potential  sources 
Reserve  capacity  for  this  work 
Capacity  in  excess  of  workload 
Availability  of  skilled  labor 
Overhead  structure 
Stability  of  labor  relationships 
Customer  knowledge 
Product  knowledge 
Repair  process  knowledge 
MRP-ll-consistent  management  methods 


The  factors  described  here  are  the  cumulative  result  of  an  extensive  literature  re¬ 
view,  several  internal  LMI  focus  groups,  and,  in  some  cases,  feedback  during  the 
trial  applications  described  in  Chapter  3.  In  the  interest  of  conciseness,  the  de¬ 
scriptions  of  these  factors  will  normally  refer  to  the  main  effects.  All  of  the  factors 
have  side  effects  of  varying  importance  that  would  be  burdensome  to  include  here. 
(We  did,  however,  include  the  side  effects  in  the  factor  weights  we  used  for 
building  the  additive  utility  model  in  Chapter  3.)  Because  the  nature  of  the  repair 
work  itself  is  the  least  abstract,  discussion  begins  with  the  factors  in  that  class. 

Nature  of  Repair  Work  Itself 

Nine  factors  related  to  the  nature  of  the  repair  work  itself  appear  in  previous  depot 
maintenance  studies  and  related  literature.  Below  we  describe  each  of  these  fac¬ 
tors  in  turn. 
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Workload  Size  vs.  Size  of  Market 

The  first  factor  considers  the  relative  size  of  the  projected  workload  compared  to 
the  total  market  for  similar  work,  including  organic  and  private-sector  repair 
workloads.  The  commercial  sector  can  be  attractive  if  it  offers  the  government  a 
large,  preexisting  infrastructure.  This  is  the  reason  why  the  Air  Force  commer¬ 
cially  supports  the  KC-10  aircraft.  Placing  one’s  requirements  in  the  general 
commercial  marketplace  is  sometimes  referred  to  as  “mainstreaming”  one’s 
requirements,  that  is,  putting  them  into  the  commercial  mainstream. 

Workload  Size  Compared  to  the  Workload  at  Sources  of  Repair 

A  related  consideration  is  the  size  of  the  projected  workload  compared  to  the 
amount  of  similar  work  at  contemplated  sources  of  repair  within  a  repair  sector.  If 
the  workload  is  comparatively  small,  then  no  matter  what  specific  source  of  repair 
it  chooses,  the  government  will  be  able  to  tap  into  a  large,  preexisting  infrastruc¬ 
ture.  In  contrast,  if  the  workload  is  large  compared  to  the  total  workload  of  a 
potential  source  of  repair  (or  is  likely  to  be  the  total  workload),  then  the  sources  of 
repair  will  have  less  flexibility  to  respond  to  unexpected  changes  in  demand, 
unless  they  deliberately  maintain  idle  capacity.  Further,  capitalization  costs  may 
increase.  Here  again  the  KC-10  fleet  provides  a  case  in  point.  It  is  a  small  fraction 
of  the  DC-10/KC-10  fleet  produced  and  serviced  by  Douglas  Aircraft  and  its 
licensees.30 

Demand  Predictability 

The  ability  to  predict  demand  is  influenced  by  both  statistical  and  state-of-the- 
world  uncertainty.  Statistical  uncertainty  is  a  fact  of  life  for  systems  with  stochas¬ 
tic  failures.  There  is  substantial  evidence  that  even  in  peacetime  the  demand  for 
complex,  high-technology  system  repair  has  considerable  statistical  uncertainty 
and  that  the  uncertainty  gets  worse  in  wartime.31  State-of-the-world  uncertainty,  of 
course,  is  particularly  challenging  for  wartime  scenarios.32  The  unpredictability  of 
demand  resulting  from  both  types  of  uncertainty  places  a  premium  on  adaptability 
of  the  logistics  chain,33  including  adaptability  of  the  repair  source.  Adaptability  of 
the  repair  source  is  what  is  at  stake  when  the  depot  maintenance  integrated  man¬ 
agement  report  describes,  as  one  of  the  response  capabilities  not  normally  avail¬ 
able  from  private  firms,  “the  ability  to  increase  output  and  change  priorities  within 
a  wide  compass  of  potential,  but  inherently  unpredictable,  needs.”34 

However,  some  workloads  are  more  predictable  than  others.  The  implication  is 
that  the  required  adaptability  of  the  repair  sector  also  varies  from  workload  to 
workload.  Thus  the  choice  of  repair  sector  should  take  into  account  the  anticipated 
demand  predictability. 
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An  evolving  design  makes  it  more  difficult  to  develop  efficient  repair  processes; 
thus  the  repair  sector  that  can  better  cope  with  this  difficulty  may  be  more  attrac¬ 
tive.  The  rate  of  change  varies  over  the  life  cycle:  it  is  generally  higher  during  a 
system’s  early  production  phase.  (Design  instability  is  one  reason  that  the  original 
equipment  manufacturer  is  often  initially  used  as  the  initial  source  of  repair.35) 

The  stability  of  the  design  affects  the  relevance  of  a  repair  sector’s  ability  to  cope 
with  design  changes. 

A  second  reason  for  potentially  including  design  stability  is  that  uncertainty  of  any 
kind  makes  it  more  difficult  to  write  a  contract.36  The  relevance  is  that,  given  a 
choice  between  an  organic  repair  facility,  an  OEM  repair  facility,  and  a  third-party 
repair  facility,  the  third  party  could  be  least  attractive  if  the  design  is  unstable.37 

Design  changes  do  not  always  affect  the  repair  process,  however.  For  purposes  of 
clarity  we  define  the  factor  “design  stability”  to  include  only  those  changes  that  do 
affect  the  repair  process. 

Technology  Renewal  Rate 

We  will  refer  to  the  long-term  rate  at  which  one  generation  succeeds  another  as 
the  technology  renewal  rate.  For  some  technology  areas,  such  as  electronics,  one 
generation  of  technology  succeeds  another  as  often  as  every  3  to  5  years.38 

Insertion  of  updated  technology  has  well-demonstrated  benefits  in  terms  of  mis¬ 
sion  performance,  reliability,  maintainability,  and  cost  reduction.39  There  is,  logi¬ 
cally,  more  opportunity  where  the  technology  renewal  rate  is  higher.  Further,  the 
ability  to  capitalize  on  new  technology  depends  in  part  on  the  source  of  repair 
(e.g.,  on  the  availability  of  an  engineering  staff).  Differences  in  engineering  capa¬ 
bility  between  one  repair  sector  and  another,  however,  may  not  be  important — 
results  will  depend  on  both  the  abilities  of  the  repair  sources  within  a  repair  sector 
and  the  renewal  rate  of  the  technologies  specific  to  the  workload  under  considera¬ 
tion.  For  these  reasons  we  include  the  technology  renewal  rate  as  a  potentially 
relevant  characteristic  of  the  workload  itself. 

Uniqueness  of  Technology 

As  described  earlier  under  the  topic  of  extraordinary  profits,  fees,  and  earnings, 
one  of  the  issues  raised  in  the  literature  on  transaction  cost  economic  theory  is 
asset  specificity.  In  general,  the  greater  the  asset  specificity,  the  more  likely  that  a 
user  will  internalize  production  rather  than  contracting  for  it.40  One  reason,  of 
course,  is  that  there  may  not  be  an  alternative — nobody  else  interested  in  the 
work.  But  even  when  there  is  an  alternative,  if  the  work  is  performed  under  con¬ 
tract  then  the  asset  specificity  can  lead  to  exploitative  behavior  such  as  charging 
high  prices.41  Firms  (and  governments)  may  decide  to  bring  work  in-house  in 
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order  to  guard  against  the  risk  of  exploitative  behavior.  The  term  “uniqueness  of 
technology”  is  an  attempt  to  capture  one  important  facet  of  asset  specificity  in 
depot  maintenance. 

Repair  Process  Dependence 

A  narrower  facet  of  asset  specificity  is  repair  process  dependence.  A  depot  main¬ 
tenance  repair  process  may  involve  significant  investment  in  skills,  equipment, 
documentation,  and  facilities  that  are  unique  to  one  or  a  few  workloads.  Other 
things  being  equal,  this  will  increase  the  cost  of  maintenance  because  of  the  initial 
nonrecurring  investment  to  establish  a  capability.  Repair  process  dependence,  be¬ 
cause  it  is  an  instance  of  asset  specificity  that  would  result  in  high  costs  to  switch 
to  an  alternative  supplier,  can  also  lead  to  exploitative  behavior.42 

Availability  of  Engineering  Data 

Acquisition  of  the  technical  information  needed  for  diagnosis  and  repair  of 
equipment  is  of  critical  importance,  because  efficient  repair  of  equipment  obvi¬ 
ously  requires  access  to  such  information.  There  are  logically  two  aspects  to  the 
engineering  data  question:  whether  the  information  exists  at  all,  and  if  so  whether 
it  is  proprietary.  This  factor  addresses  whether  the  engineering  information  exists. 

Absence  of  Proprietary  Data  Issues 

Even  if  the  technical  data  exist,  they  will  not  necessarily  be  equally  available  to 
organic  and  commercial  sources  of  repair  if  the  original  designer  asserts  proprie¬ 
tary  (data  rights)  claims.  This  issue  is  of  demonstrated  practical  importance  to  the 
repair  sector  selection  decision — as  the  Air  Force  found  when  it  privatized  the 
Aerospace  Guidance  and  Metrology  Center  at  Newark  Air  Force  Station.43 

The  problem  is  that  data  rights  affect  third-party  commercial  sources  of  repair 
more  than  government  sources.  The  original  designers  of  equipment  are  generally 
more  willing  to  provide  government  depots  access  to  their  proprietary  data  than 
they  are  other  commercial  firms,  because  the  government  is  unlikely  to  be  a 
manufacturing  competitor.  Even  in  the  absence  of  data  rights  claims,  however,  the 
OEM  will  likely  gain  a  “first-mover”  advantage  over  a  third-party  source  of 
repair.  The  OEM  possesses  the  engineering  knowledge  that  permitted  developing 
the  data  in  the  first  place. 

Nature  of  Relationship  with  Sources 

The  second  category  of  factors  includes  those  that  reflect  the  relationship  between 
the  government  and  the  potential  sources  of  repair.  Representative  factors  in  this 
category  include  the  type  of  contract  (or  equivalent  administrative  arrangement) 
between  the  parties;  government  policies  on  such  considerations  as  lot  size  and 
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sources  of  repair  parts;  and  the  incentive  structure.  We  have  identified  six  such 
factors. 

Type  of  Relationship 

As  noted  earlier,  one  of  the  primary  arguments  for  maintaining  organic  depot 
maintenance  is  to  maintain  the  ability  to  “immediately  increase  output  and  change 
priorities  within  a  wide  compass  of  potential,  but  inherently  unpredictable  needs.” 

Two  assumptions  are  built  into  this  statement.  The  first  is  that  there  is  a  require¬ 
ment  to  “immediately  increase  output  and  change  priorities.”  This  requirement 
stems  from  the  large  uncertainty  associated  with  conflict — Clausewitz’s  “fog  of 
war.”44  Hence  the  customer’s  requirement  for  flexibility — to  be  able  to  rapidly 
change  priorities  in  order  to  respond  to  uncertain  wartime  demand — is  probably 
uncontestable. 

The  second,  unstated,  assumption  is  that  only  organic  sources  can  satisfy  the  re¬ 
quirement;  and  behind  that  assumption  is  a  third:  that  the  flexibility  expected  in  a 
relationship  with  an  organic  depot  makes  it  is  easier  to  change  requirements  than 
with  a  commercial  source.  These  assumptions  find  support  in  transaction  cost  the¬ 
ory  and  the  literature  on  supply  chain  management,  which  show  that  it  is  generally 
harder  and  takes  longer  to  provide  for  contingent  performance  through  a  contract 
mechanism  than  through  a  command  mechanism. 

The  fact  that  it  is  generally  harder  and  takes  longer  does  not  mean  that  it  is  always 
so.  After  all,  as  a  result  of  information  technology  such  as  electronic  data  inter¬ 
change,  the  cost  per  transaction  and  speed  of  transactions  between  trading  partners 
are  rapidly  falling.45  For  these  reasons,  the  relative  ability  to  rapidly  change  re¬ 
quirements  needs  to  be  looked  at  on  a  case-by-case  basis.  Thus,  it  is  a  legitimate 
factor  to  be  considered  when  deciding  between  repair  sectors. 

Scope  of  Relationship 

If  it  is  important  to  be  able  to  levy  contingent  requirements  on  a  source  of  repair, 
then  it  is  obviously  important  for  the  source  to  have  the  flexibility  to  respond  to 
changes  in  requirements.  This  is  particularly  the  case  where  the  repair  volumes  for 
individual  components  (most  aircraft  components,  for  example)  are  very  low. 

One  determinant  of  flexibility  is  the  scope  of  the  relationship  (illustrated,  for  in¬ 
stance,  by  the  number  of  different  types  of  components  included).  The  broader  the 
scope,  the  more  the  peaks  and  valleys  from  individual  components’  demand  pat¬ 
terns  will  tend  to  level  each  other  out  when  viewed  in  aggregate  at  a  source  of 
repair.  If  the  individual  components  also  use  complementary  processes  and 
resources  (similar  skills,  similar  equipment,  similar  task  sequences,  etc.),  then  the 
source  of  repair  can  switch  from  repair  of  one  component  to  another  to  respond  to 
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a  change  in  requirements.46  Scope  can  be  either  vertical  (many  components  from 
the  same  system)  or  horizontal  (similar  components  from  many  systems). 

Because  government  depots  are  generally  large,  integrated  facilities  but  commer¬ 
cial  repair  sources  may  or  may  not  be,  this  is  a  potentially  relevant  factor. 

Policy  on  Related  Functions 

The  previous  factor  addressed  the  scope  of  the  relationship  in  the  context  of  how 
many  components  are  repaired.  Another  more  particular  facet  of  that  scope  is  the 
breadth  of  functional  responsibility.  The  department,  when  contracting  for  depot 
repair,  can  hold  the  source  of  repair  responsible  for  just  repair  or  can  broaden  the 
responsibility  to  include  other  activities,  such  as  sustaining  engineering  and  mate¬ 
riel  management. 

Low-cost  production  of  a  heterogeneous  workload  (i.e.,  achieving  economy  of 
scope)  depends,  in  part,  on  broadening  functional  responsibility.47  As  an  example, 
the  Coast  Guard  originally  contracted  with  Lycoming/Textron  for  repair  of 
LTS101  engine  parts  using  a  traditional  repair-only  contract.  By  converting  this 
contract  to  power-by-the-hour  (where  a  vendor  is  reimbursed  based  on  the  number 
of  operating  hours  produced)  and  placing  responsibility  with  Lycoming/Textron 
for  engineering,  rotable  spares,  and  transportation,  the  Coast  Guard  decreased  the 
shop  visit  rate  by  a  factor  of  four  and  simultaneously  reduced  the  annual  support 
cost  50  percent.48  The  same  considerations  apply  even  if  repair  is  performed  or¬ 
ganically.  There  is  evidence,  for  instance,  that  where  materiel  management  and 
engineering  responsibility  are  separated  the  product  improvement  process  essen¬ 
tially  stops.49 

The  results  described  in  the  previous  paragraph  should  not  be  a  surprise.  These  are 
integrated  logistics  first  principles — one  gets  better  results  when  all  of  the  impor¬ 
tant  facets  of  a  system  are  considered  on  an  integrated  basis.  Thus,  responsibility 
for  functions  related  to  repair  is  a  factor  that  should  be  considered. 

Policy  on  Lot  Sizes 

The  repair  lot  size  has  a  significant  effect  on  flexibility  to  respond  to  changes  in 
requirements.50  Large  lot  sizes  impair  flexibility.  In  general,  the  operative 
Department  of  Defense  policy  has  been  and  still  is  to  batch  unserviceable  compo¬ 
nents  prior  to  shipment  to  either  a  commercial  or  an  organic  source  of  repair,  thus 
generating  relatively  large  lot  sizes.51  However  this  is  not  necessarily  a  static 
policy  and  will  not  necessarily  be  uniformly  applied  across  the  commercial  and 
organic  sectors  in  the  future.  Thus  it  a  candidate  factor  to  be  considered. 
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The  time  spent  awaiting  parts  is  a  major  contributor  to  total  flow  time  at  both 
commercial  and  organic  repair  facilities.52  In  part  this  is  because  department  pol¬ 
icy  restricts  how  repair  parts  are  obtained.  Chenowith  and  Abell  illustrate: 

The  government  purchases  and  manages  many  parts,  commonly  referred 
to  as  government  furnished  material  (GFM),  and  is  often  the  source  of 
supply.  When  that  occurs,  a  Military  Standard  Requisitioning  and  Issue 
Procedures  (MILSTRIP)  action  takes  place.  In  the  event  that  the  gov¬ 
ernment  discovers  it  cannot  fulfill  the  request  and  additional  supplies  are 
not  expected  for  some  time,  it  gives  the  contractor  permission  to  pur¬ 
chase  the  material  on  its  own  with  Air  Force  funds.53 

The  coordination  process  for  a  contractor  (or  organic  facility)  to  obtain  permission 
to  exit  the  MILSTRIP  process  takes  time.  There  are  even  cases  where  repair  parts 
purchased  for  one  program  cannot  be  used  for  another.  When  this  happens  the 
source  of  repair,  contractor  or  organic,  might  have  to  requisition  parts  that  it  al¬ 
ready  has  on  hand. 

Policies  such  as  these,  based  on  notions  of  accountability  and  a  priori  prioritiza¬ 
tion,  have  the  significant  disadvantage  of  extending  flow  times.  It  also  stands  to 
reason  that  the  delivery  uncertainty  they  introduce  will  increase  variability  in  flow 
times.  By  contrast,  a  policy  that  provides  the  flexibility  to  obtain  parts  from  the 
most  responsive  sources  will  reduce  flow  times  and  reduce  variability.  This 
thinking  is,  of  course,  part  of  the  rationale  for  direct-vendor  delivery  programs  in 
the  commercial  sphere  and  at  the  Defense  Logistics  Agency. 

More  important  for  our  purposes,  repair  parts  policies  will  have  an  effect  when 
they  are  more  flexible  for  one  source  of  repair  than  for  another.  For  this  reason, 
the  policy  on  repair  parts  is  a  factor  to  be  considered  when  selecting  the  repair 
sector. 

Incentive  Structure 

Incentives  and  incentive  structures  figure  in  just  about  every  report  on  depot 
maintenance.  It  should  not  be  necessary  to  provide  a  substantial  justification  that 
incentives  to  reduce  cost  and  improve  productivity  can  make  a  difference.  Here, 
our  interest  is  on  how  the  incentive  structure  the  government  puts  in  place  affects 
the  characteristics  of  repair  sectors  (as  described  earlier),  rather  than  on  the  par¬ 
ticulars  of  specific  contract  incentives.  The  incentive  structure  can  be  either  inten¬ 
tional  or,  especially  with  organic  sources,  de  facto. 

Of  particular  importance  is  the  difference  between  the  incentive  structure  under 
competition  versus  a  sole  source.  The  monopolist’s  bargaining  power  can  create 
distorted  incentives — such  as  maximizing  budgets  and  serving  internal  interests — 
rather  than  reducing  costs,  raising  productivity,  and  serving  the  customer’s 
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interest.54  The  potential  for  distorted  incentives  when  the  government  is  bound  to 
a  single  commercial  provider  is  generally  recognized.  Not  as  generally  recognized 
is  the  fact  that  organic  providers  can  also  hold  a  monopoly  position.55 

Even  where  formal  competition  is  absent,  it  is  possible  to  get  some  of  the  effects 
of  competition  if  a  sole-source  provider  can  be  disciplined  by  the  potential  entry 
of  a  competitor.56  Thus,  from  the  standpoint  of  incentive  structure,  creating  or 
retaining  an  organic  capability  may  have  merit  if  the  only  alternative  is  a  commer¬ 
cial  sole  source.  Obviously,  the  reverse  argument  also  applies:  using  a  commercial 
source  may  have  merit  if  the  only  alternative  is  a  government  source. 

Nature  of  Sources 

The  third  category  of  factors  that  influence  the  characteristics  of  the  sources  of 
repair  is  directly  related  to  the  nature  of  the  commercial  and  organic  sources  of 
repair  themselves.  We  identified  12  factors  in  this  category. 

Availability  of  More  Than  One  Source 

The  reasons  for  considering  the  availability  of  more  than  one  source  are  the  same 
as  those  presented  above  with  regard  to  incentive  structure.  The  bargaining  power 
a  sole  source  wields  can  create  distorted  incentives — such  as  maximizing  budgets 
and  serving  internal  interests — rather  than  reducing  costs,  raising  productivity, 
and  serving  the  customer’ s  interest.  As  we  discussed  previously,  both  commercial 
and  organic  sources  can  be  in  a  monopoly  position.  Not  surprisingly,  it  was  clear 
from  our  interviews  with  field  personnel  that  the  availability  of  more  than  one 
source  was  of  primary  importance  in  repair  sector  selection  decisions. 

Availability  of  Sources  with  Complementary  Workload 

Complementary  workload  is  workload  that  requires  equipment,  training,  and  skills 
similar  to  the  workload  under  consideration.  The  reasons  for  including  this  factor 
are  in  part  the  same  as  those  presented  earlier  with  regard  to  the  scope  of  the  rela¬ 
tionship.  If  repair  sources  have  other,  complementary  workload,  they  are  in  a  bet¬ 
ter  position  to  accommodate  the  peaks  and  valleys  of  individual  component 
demand  patterns. 

In  addition,  and  potentially  more  importantly,  sources  with  complementary  work¬ 
load  are  able  to  spread  the  fixed  costs  of  equipment  and  training  over  a  wider 
workload  base.  In  the  absence  of  complementary  workload,  the  customer  user 
bears  the  entire  cost. 

Interest  by  Potential  Sources 

Interest  by  potential  sources  is  important  from  two  standpoints.  First,  if  there  are 
no  interested  commercial  sources  (as  can  be  the  case  with  obsolete  technologies), 


2-19 


then  the  government  must  either  do  the  repair  itself  or  insert  new  technology 
through  redesign. 

Second,  a  single  interested  source  can  exercise  the  monopoly  bargaining  power 
described  above.  Having  multiple  interested  sources  hedges  against  this  possibil¬ 
ity. 

Reserve  Capacity  for  This  Work 

Capacity  is  the  amount  of  workload  that  a  facility  can  effectively  turn  out  while 
producing  the  product  mix  that  the  facility  is  designed  to  accommodate.57  If  a 
source  of  repair  is  already  producing  at  capacity,  then  it  will  necessarily  have  to 
make  a  capital  investment  to  provide  additional  capacity.  Other  things  being 
equal,  a  repair  sector  comprising  sources  of  repair  that  do  not  have  to  make  this 
investment  will  produce  at  lower  cost,  since  its  sources  of  repair  would  not  have 
to  amortize  the  investment. 

Capacity  In  Excess  of  Workload 

The  previous  factor  addressed  the  availability  of  reserve  capacity.  However, 
capacity  is  a  two-edged  sword.  Unutilized  capacity  increases  overhead  costs — a 
continuing  problem  for  both  the  organic  depots  and  their  commercial  counter¬ 
parts.58  Because  there  are  two  different  capacity-related  considerations,  which  act 
in  opposing  ways,  we  have  provided  a  second  factor,  capacity  in  excess  of  work¬ 
load. 

Availability  of  Skilled  Labor 

Availability  of  the  right  personnel  quantities  and  skills  is  a  well  recognized  and 
fairly  obvious  requirement  for  maintenance  activities.  It  is  of  sufficient  importance 
for  new  systems  that  personnel  training  is  a  logistics  element  in  its  own  right. 
Thus,  the  degree  to  which  skilled  labor  is  already  available  is  a  legitimate  factor 
when  comparing  one  source  of  repair  to  another. 

Overhead  Structure 

Potential  sources  of  repair  can  differ  significantly  in  their  overhead  structures  and, 
as  a  result,  their  indirect  costs.  For  example,  original  equipment  manufacturers 
carry  large  engineering  staffs.  As  a  result,  if  the  engineering  staffs  are  in  the  same 
overhead  pool  as  that  applied  to  maintenance,  then  they  will  have  higher  rates 
than  contractors  whose  overhead  pools  are  more  narrowly  focused  on  mainte¬ 
nance.  For  this  reason,  overhead  structure  is  included  as  a  relevant  characteristic 
of  the  repair  sectors. 

At  least  two  points  should  be  clarified.  First,  organic  depots  in  recent  years  have 
experienced  significant  overhead  rate  increases,  but  the  reasons  have  more  to  do 
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with  excess  capacity,  already  provided  for  above,  than  with  overhead  structure  in 
the  sense  meant  here.59  Second,  it  would  be  incorrect  to  conclude  that  OEMs  al¬ 
ways  have  high  overhead  rates.  It  is  not  unusual  for  OEMs  to  create  independent 
service  companies  specifically  designed  to  have  low  overhead  rates. 

Stability  of  Labor  Relationships 

As  discussed  earlier  in  this  chapter,  an  important  reason  why  DoD  managers  tend 
to  prefer  organic  sources  is  the  perceived  risks  in  using  commercial  sources.  One 
mentioned  often  (both  informally  and  in  formal  reports)  is  the  possibility  that 
strikes  might  interfere  with  production.  Because  of  this  concern,  we  include  sta¬ 
bility  of  labor  relations  as  a  factor.60 

Customer  Knowledge 

One  of  the  innovative  practices  characteristic  of  successful  private-sector  firms  is 
a  continual  effort  to  know  the  customer  better  than  the  competition  does  61  By 
customer  knowledge  we  mean  the  detailed  understanding  needed  to  capture  cus¬ 
tomer  requirements  and  translate  them  into  performance  meaningful  to  the  cus¬ 
tomer.  For  maintenance,  the  detailed  understanding  that  is  customer  knowledge 
takes  the  form  of  both  customer  production  requirements  and  engineering  knowl¬ 
edge  (such  as  failure  modes  and  effects)  of  the  customer’s  equipment. 

Despite  expectations  to  the  contrary,  organic  depots  may  not  necessarily  be  in  a 
privileged  position  to  have  such  customer  knowledge.62  Because  of  the  structural 
relationships  between  depot  production  and  the  DoD  materiel  management  sys¬ 
tems,  the  customer  of  the  depot  is  the  item  manager  more  so  than  the  field  user. 
Among  the  longstanding  results  are  significant  gaps  between  what  is  in  work  in 
DoD  depots  and  what  customers  actually  need.63 

Product  Knowledge 

The  specialized  knowledge  of  the  product  itself,  like  knowledge  of  the  process,  is 
a  qualifying  factor  that  tends  to  favor  the  incumbent  source  of  repair.  Product 
knowledge  includes  such  information  as  product  history  and  histories  of  sister 
products;  details  of  the  tradeoffs  that  were  made  during  product  design;  details  of 
manufacturing  processes;  failure  history  and  failure  modes;  as  well  as  historical, 
present,  and  planned  logistical  support  methods. 

Repair  Process  Knowledge 

The  specialized  knowledge  of  repair  processes  needed  to  accomplish  maintenance 
is  a  natural  barrier  to  entry  into  the  market  for  weapon  system  component  repair. 
Thus,  in  choosing  a  repair  sector,  the  availability  of  this  process  knowledge  is  a 
relevant  factor.  Generally,  of  course,  it  is  a  factor  that  would  tend  to  favor  incum¬ 
bents,  whether  organic  or  contract. 
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In  recent  years,  as  part  of  its  effort  to  modernize  automation,  the  DoD  depot 
maintenance  community  has  defined  a  set  of  preferred  management  methods  in  an 
improved  functional  baseline.64  These  preferred  methods  mirror  improvements  in 
the  commercial  manufacturing  world  that  have  permitted  successful  competition 
in  a  market-driven  environment.  The  short-hand  term  for  the  improved  methods  is 
manufacturing  resources  planning  (MRP-II).65  The  fairly  lengthy  list  of  improved 
management  methods  that  makes  up  MRP-II  includes 

♦  multipurpose  structured  bills  of  material  with  individual  item  tracking 
capability; 

♦  automatic  consideration  of  engineering  change  notices; 

♦  automatic,  time-phased,  planned  shop  and  material  orders; 

♦  incorporation  of  priority  factors  during  scheduling; 

♦  item-level  planning,  tracking,  history,  and  rescheduling; 

♦  continual  recalculation  of  resource  requirements  based  on  planning 
schedules,  detailed  bills  of  material,  and  activity; 

♦  automatic  reprioritization  based  on  changes  in  customer  requirements 
(deexpediting);  and 

♦  linkage  of  capacity  requirements  planning  to  time-phased  resource 
scheduling. 

The  evidence  is  compelling  that  absence  of  such  improved  methods  (particularly 
the  inability  to  reprioritize  based  on  changes  in  customer  requirements)  results  in 
long  repair  cycle  times,  large  in-process  inventories,  and  misapplied  resources,  in 
both  the  organic  depot  maintenance  sector  and  in  DoD’s  supporting  contractors.66 

Because  of  the  success  of  MRP-II  methods  in  manufacturing;  demonstrated  bene¬ 
fits  of  MRP-II-like  methods  in  repair  (e.g.,  remanufacturing);  and  less  than  satis¬ 
factory  results  in  the  absence  of  these  methods,  availability  of  MRP-II  methods  is 
a  legitimate  factor. 

Summary  of  Elements  in  the  New 
Decision  Model 

This  chapter  has  described  a  conceptual  decision  model  with  three  levels.  Table  2-1 
summarizes  the  elements  at  each  level. 
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Table  2-1.  Elements  of  Decision  Model  at  Each  Level 


Customer  needs 

Characteristics  of 
sources  of  repair 

Factors  that  determine 
the  characteristics 

Ability  to  meet  basic  demand 

Agility  and  responsiveness  in 
the  face  of  changing  demand 

Quality  product  and  service 

Low  total  cost 

Continuous  improvement  of 
both  product  and  service 

Adequate  capability  and 
capacity 

Scope  and  scale  economy 

Overhead  costs 

Non-value-added  work 

Extraordinary  profits,  fees,  and 
earnings 

Responsiveness  to  changing 
needs 

•  Reprioritization  capability 

•  Surge  capability 

•  Task  flexibility 

•  Customer  linkage 

•  Administrative  ease 

Incentive  to  improve 

Work  stoppage  protection 

Pipeline  size 
(quantity  and  time) 

•  Order  and  ship  (admin.) 

•  Repair  cycle  at  source  of 
repair  (SOR) 

•  Material  movement 
(user  to  SOR  and  return) 

Output  quality 

•  Of  product 

•  Of  service 

Nature  of  repair  work  itself 

•  Workload  size  vs.  size  of  mar¬ 
ket 

•  Workload  size  compared  to 
the  workload  at  sources  of 
repair 

•  Demand  predictability 

•  Design  stability 

•  Technology  lifetime 

•  Uniqueness  of  technology 

•  Repair  process  dependence 

•  Availability  of  engineering  data 

•  Absence  of  proprietary  data 
issues 

Nature  of  relationship  with  sources 

•  Type  of  relationship  (flexible 
vs.  inflexible) 

•  Scope  of  relationship 

•  Policy  on  related  functions 

•  Policy  on  lot  sizes 

•  Policy  on  repair  parts 

•  Incentive  structure 

Nature  of  sources 

•  Availability  of  more  than  one 
source 

•  Availability  of  sources  with 
complementary  workload 

•  Interest  by  potential  sources 

•  Reserve  capacity  for  this  work 

•  Capacity  in  excess  of  workload 

•  Availability  of  skilled  labor 

•  Overhead  structure 

•  Stable  labor  relationships 

•  Customer  knowledge 

•  Product  knowledge 

•  Process  knowledge 

•  MRP-ll-consistent  manage¬ 
ment  methods 
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Operationalizing  the  Model 


The  basic  logic  of  the  decision  model,  as  illustrated  earlier  in  Figure  1-1,  is  that 
the  extent  to  which  each  repair  sector  can  satisfy  customers  depends  on  the  char¬ 
acteristics  of  the  repair  sectors.  Those  characteristics  depend,  in  turn,  on  the  nature 
of  the  repair  work,  the  nature  of  the  relationship  the  government  has  with  its  re¬ 
pair  sources,  and  certain  inherent  factors  related  to  the  sources  themselves, 
whether  public  or  private. 

We  operationalize  this  model  in  a  simple  mathematical  form  in  Appendix  A.  Our 
approach  depends  on  establishing  evaluation  criteria,  weighting  the  criteria,  and 
evaluating  RSS  alternatives  against  them  to  determine  which  alternative  scores  the 
highest.  Appendix  B  describes  a  technique  for  interpreting  the  scores  in  terms  of 
relative  risk. 

The  mathematical  form  described  in  Appendix  A  and  the  method  for  interpreting 
results  described  in  Appendix  B  are  consistent  with  those  used  in  DoD  procure¬ 
ment  source-selection  procedures.  For  that  matter,  so  is  the  whole  concept  of  es¬ 
tablishing  criteria,  weighting  the  criteria,  and  evaluating  alternatives.67  A  major 
reason  for  taking  this  approach  is  that  if  the  basic  procedure  is  acceptable  for  pro¬ 
curements  generally,  then  it  logically  also  ought  to  be  acceptable  for  a  close  ana¬ 
logue,  the  repair  sector  selection  decision. 

Limitations  of  the  Model 

The  evaluation  approach  described  in  this  report  focuses  on  the  requirements  of 
the  depot  maintenance  customer  and  the  ability  of  a  sector  to  satisfy  them.  In  tak¬ 
ing  this  approach,  we  have  excluded  a  number  of  potentially  important  considera¬ 
tions: 

♦  Equity  considerations  (for  example,  effects  on  the  government  or  contract 
workforce  of  shifting  workload  from  one  to  the  other). 

♦  Interaction  with  overall  program  acquisition  strategy.  For  example,  if  a 
program  is,  for  other  reasons,  considering  interim  contract  support,  that 
decision  and  the  RSS  decision  probably  should  be  taken  together. 

♦  Affordability  within  budget.  It  is  possible  that  the  preferred  choice  of  sec¬ 
tor  would  simply  not  be  affordable  within  the  available  budget  if  it  re¬ 
quires  large  up-front  investments. 

♦  Political  considerations. 
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Additionally,  because  the  approach  is  inherently  incremental,  workload-by- 
workload,  it  does  not  address  issues  relating  to  long-term  planning  regarding 
depot  maintenance. 
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Chapter  3 

The  Repair  Sector  Selection  Decision  Process 


To  reach  public  versus  private  repair  sector  selection  decisions  that  are  consis¬ 
tently  in  the  best  interest  of  the  department  requires  a  process  that  incorporates 

♦  a  decision  model — i.e.,  a  representation  of  the  various  elements  of  the  RSS 
decision  and  their  interrelationships — that  captures  most  of  what  is  im¬ 
portant  to  the  decision; 

♦  a  method  for  bringing  to  bear  reasonably  complete  and  unbiased  data  on 
both  the  public  and  private  sectors; 

♦  a  procedure  for  interpreting  the  output  of  the  decision  model;  and 

♦  a  provision  for  executive-level  holistic  evaluation. 

This  last  provision  recognizes  that  a  model  is  an  aid  to  executive  judgment,  not  a 
substitute  for  it.  We  include  provision  for  holistic  evaluation  to  guard  against  at¬ 
tributing  more  credence  than  appropriate  to  the  precise-appearing  scores  that  a 
decision  model  can  yield.  Although  the  RSS  decision  model  incorporates  most  of 
the  relevant  decision  elements  in  a  reasonable  way,  there  will  always  be  important 
externalities.  An  explicit  model  such  as  described  in  Chapter  2  is  a  simplification 
of  a  deeper  and  more  complete  tacit  understanding.1  For  instance,  although  we 
identified  some  of  the  potential  externalities  at  the  conclusion  of  Chapter  2,  others 
may  be  present  as  well.  Further,  since  the  input  data  are  provided  by  people  and 
reflect  their  individual  judgments,  a  common-sense  test  of  overall  reasonableness 
is  warranted  in  any  event. 

It  was  clear  from  previous  work  related  to  the  RSS  decision  and  from  the  inter¬ 
views  we  conducted  that  the  existing  processes  do  not  satisfy  the  four  needs  listed 
above,  particularly  the  first  two.  The  problems  with  the  existing  decision  models 
(decision  tree,  life-cycle  cost,  CORE)  were  described  in  Chapter  2.  Serious  prob¬ 
lems  also  exist  in  the  way  the  inherently  judgmental  input  data  are  obtained.  As 
will  be  discussed  more  completely  below,  such  judgments  are  subject  to  numerous 
sources  of  intentional  and  unintentional  bias.  We  do  not  see  provisions  in  the  pre¬ 
sent  processes  to  guard  against  such  bias.  For  instance,  by  failing  to  include  pri¬ 
vate-sector  personnel  in  the  existing  methods  it  is  highly  likely  (and  corroborated 
by  this  study,  in  Chapter  4)  that  the  government’s  understanding  of  the  private 
sector  is  biased,  simply  because  it  is  incomplete.  Similarly,  relying  on  government 
personnel  who  may  have  a  stake  in  the  outcome  will  tend  to  produce  bias,  no 
matter  how  good  their  intentions. 
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To  address  problems  with  the  existing  decision  approaches,  we  designed  a  revised 
four-step  process  to  correspond  to  the  four  necessary  components  (Figure  1-2). 
Chapter  2  described  the  first  component,  the  proposed  new  RSS  decision  model. 
This  chapter  describes  the  second  and  third  components.  Discussion  begins  with 
step  2,  obtaining  input  data.  It  then  continues  with  the  process  for  interpreting  re¬ 
sults.  We  do  not  describe  development  of  a  briefing  for  presentation  to  the  pro¬ 
gram  manager  or  other  person  designated  to  make  the  RSS  decision,  which  is 
assumed  to  follow  normal  administrative  practice.  Neither  do  we  describe  the  ex¬ 
ecutive  level,  holistic  evaluation  itself,  which  is  assumed  to  be  integrative  and  un¬ 
structured. 

Eliciting  Input  Data 

Three  problems  need  to  be  solved  in  eliciting  the  input  information.  First,  the  in¬ 
formation  needed  for  a  rational  decision  is  initially  dispersed  among  many  indi¬ 
viduals  and  probably  disparate  organizations,  rather  than  integrated.  Hence,  there 
has  to  be  some  means  for  assembling  and  integrating  the  information.  The  second 
problem  is  guarding  against  bias,  and  the  third  is  providing  for  reasonable  consis¬ 
tency  from  one  RSS  evaluation  to  another. 

Joint  Evaluation 

To  address  these  three  problems,  we  designed  a  data  elicitation  method  as  a  com¬ 
panion  to  the  new  decision  model.  The  method  relies  on  eliciting  the  data  in  an 
integrated  product/process  team  setting.  The  IPT  members  comprise  system 
program  office  personnel  (both  technical  and  contract  management),  DoD  depot 
representatives,  and  industry  representatives. 

The  IPT-based  method  involves  assembling  personnel  with  knowledge  of  the  is¬ 
sues  important  to  the  repair  sector  selection  decisions,  as  described  above;  pro¬ 
viding  an  introductory  familiarization  briefing  on  the  purpose  of  the  evaluation; 
and  having  the  members  of  the  IPT  collectively  determine  the  scores  for  each 
repair  sector  on  each  of  the  factors  under  consideration. 

Addressing  Bias 

The  substantial  literature  on  eliciting  judgments  indicates  that  judgments  are  sub- 
ject  to  numerous  sources  of  intentional  and  unintentional  bias.  Intentional  bias  is 
usually  attributed  to  motivational  issues,  and  unintentional  bias  to  cognitive  limi¬ 
tations. 
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Our  review  of  the  relevant  literature  indicates  that  potentially  relevant  inten¬ 
tional/motivational  biases  include 

♦  outright  selfishness  and  guile; 

♦  wishful  thinking  (responding  based  on  hoped-for  results); 

♦  impression  management  (“What  would  my  organization  think  of  my 
answer?”); 

♦  misinterpretation  (tacitly  assuming  a  meaning  based  on  experience  with 
similar  sounding  terms);  and 

♦  misrepresentation  (making  an  invalid  assumption  about  a  probability  dis¬ 
tribution — such  as  ignoring  low-likelihood  events  or  considering  only  a 
few  possible  outcomes — when  encoding  a  response). 

Similarly,  relevant  cognitive  biases  include 

♦  failure  to  take  into  account  base  rates  (ignoring  the  lessons  of  experience 
and  history); 

♦  anchoring  without  adequate  adjustment  (for  example,  attaching  too  much 
importance  to  a  particular,  generally  recent,  event  with  which  a  person  is 
familiar); 

♦  inconsistency  (forgetting  an  assumption  made  earlier  and  contradicting  it); 

♦  availability  (recalling  familiar,  concrete,  or  recent  events  and  overestimat¬ 
ing  the  frequency  of  similar  events); 

♦  underestimation  of  uncertainty  (resulting  in  failure  to  account  for  the 
actual  amount  of  uncertainty  in  answers  given);  and 

♦  failure  to  seek  evidence  to  the  contrary. 

To  provide  some  protection  against  these  sources  of  bias,  we  incorporated  both 
“direct”  elicitation  and  focus  group  techniques  in  the  elicitation  method.  Direct 
elicitation  was  implemented  in  the  form  of  a  structured,  workbook-based  ques¬ 
tionnaire.  The  IPT  naturally  constitutes  a  focus  group  and  in  so  doing  gains  the 
advantage  of  focus  group  pooled  judgments.3  Table  3-1  details  how  we  sought  to 
mitigate  each  of  the  bias  problems. 
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Table  3-L  Sources  of  Bias  and  Mitigation  Methods 


Potential  source  of  bias 

Method  used  to  counter  bias 

Selfishness  and  guile 

Included  experts  from  both  government  and  industry 
repair  sectors  to  provide  checks  and  balances  on  each 
other 

Wishful  thinking 

Included  experts  from  both  government  and  industry 
repair  sectors 

Impression  management 

Included  experts  from  both  government  and  industry 
repair  sectors 

Misinterpretation 

Used  common  terminology,  and  pretested  the 
terminology  to  guard  against  misinterpretation 

Provided  an  evaluation  workbook  with  defined  terms 

Explained  terms  as  needed  during  the  elicitation 

Misrepresentation 

Provided  background  information  on  expected 
distributions  for  each  factor 

Failure  to  take  base  rates 
into  account 

Provided  background  information  on  expected 
distributions  for  each  factor 

Anchoring  without 
adequate  adjustment 

Group  performed  the  assessments,  rather  than 
individuals 

Inconsistency 

Grouped  and  limited  the  number  of  questions 

Provided  a  workbook  to  each  evaluator  with  space  to 
make  notes 

Provided  standardized  rating  scales  with  defined 
anchors 

Availability 

Included  experts  from  multiple  functional  areas  (e.g., 
depot  maintenance,  program  management,  contract 
management;  from  both  government  and  industry) 

Underestimation  of 
uncertainty 

Permitted  range  rather  than  point  responses 

Did  not  insist  on  consensus  answers;  difficulty  in 
reaching  consensus  was  taken  as  an  indicator  of  un¬ 
certainty  and  a  need  for  sensitivity  analysis 

Failure  to  seek  evidence  to 
the  contrary 

Included  experts  from  both  government  and  industry 
repair  sectors 

Consistency  Among  Evaluation  Sessions 

To  encourage  consistency  of  scoring  between  one  evaluation  session  and  another, 
we  developed  a  set  of  rating  scales  and  published  them  in  an  evaluation  workbook 
used  during  the  trial  evaluations.  An  example  rating  scale  is  illustrated  in 
Figure  3-1 ;  the  workbook  is  in  Appendix  C. 
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Figure  3-1.  Example  Rating  Scale 

Relative  size  (amount)  compared  to  marketplace 


1  3  5  7  9 
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The  rating  scale  comprises  three  parts.  The  first  is  a  statement  of  the  dimension 
being  evaluated.  The  second  is  a  line  from  worst  assessment  to  best  assessment 
with  five  marked  points  on  the  line  (an  interval  scale).  The  third  part  is  a  set  of 
defined  anchors  for  particular  points  on  the  line.  All  of  the  rating  scales  have  at 
least  three  defined  anchors,  one  each  for  the  worst  (1),  typical  (5),  and  best 
(9)  evaluations.  Some,  such  as  the  scale  illustrated  in  Figure  3-1,  also  have  the 
(3)  and/or  (7)  points  defined.  We  did  not  attempt  to  provide  a  definition  for  a 
point  where  the  definition  would  have  been  more  forced  than  useful,  such  as  for 
(7)  in  Figure  3-1.  The  scale  illustrated  is  a  version  of  the  classical  Likert  method, 
with  both  numerical  and  verbal  labels  for  the  categories  of  answers.4  The  reason 
for  including  both  numerical  and  verbal  labels  is  that  people  prefer  to  communi¬ 
cate  uncertainty  verbally  but  receive  it  numerically,  while  the  accuracy  of  the  two 
different  response  modes  is  essentially  the  same.5 

Interpreting  Results 

When  the  scores  comparing  two  sectors  are  input  into  the  mathematical  model 
(described  in  Appendix  A),  the  results  look  like  Table  3-2. 

Such  a  display  establishes  that  repair  Sector  B  scored  overall  higher  than  repair 
Sector  A  (0.321  versus  0.312).  It  also  indicates  what  differences  at  the  customer- 
need  level  contributed  to  the  higher  score.  For  example,  the  customer  need  for 
“quality  product  and  service”  differs  by  0.019,  whereas  both  sectors  have  nearly 
the  same  score  for  “surge.” 
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Table  3-2.  Example  Results  from  Mathematical  Model 


Factor 

Subfactor 

Sector  A 

Sector  B 

Overall 

comparison 

0.312 

0.321 

Meet  basic  demand 

0.337 

0.340 

Agility/responsiveness:  Reprioritize 

0.485 

0.509 

Surge 

0.406 

0.405 

Adapt 

0.236 

0.236 

Quality  product  and  service 

0.357 

0.376 

Low  total  cost:  Of  maintenance,  per  se 

0.352 

0.377 

Of  administration 

0.253 

0.260 

Of  capitalization 

0.190 

0.193 

Continuous  improvement  of  quality  and  service 

0.192 

0.197 

What  such  a  display  does  not  do  is  provide  insight  into  the  significance  of  overall 
scores  like  0.312  and  0.321.  Without  additional  information,  we  would  not  know 
how  meaningful  the  difference  is,  nor  would  we  know  whether  either  sector  poses 
a  low  risk  to  satisfaction  of  customer  needs,  an  average  risk,  or  a  high  risk. 

An  approach  to  establishing  the  meaning  of  these  scores  is  to  recognize  that  if  the 
results  of  many  such  analyses  were  available,  then  the  pattern  of  those  results 
would  form  a  backdrop  against  which  the  current  analysis  (Table  3-2)  could  be 
viewed.  However,  we  are  describing  a  new  methodology  and  the  results  of  many 
such  analyses  are  not  available.  (Further,  even  if  the  methodology  had  been  in 
place  for  some  time,  prior  results  would  not  necessarily  be  available.) 

Nonetheless,  one  can  artificially  create  a  backdrop  of  “results”  by  simulating  the 
performance  of  a  large  number  of  analyses.  The  distribution  of  results  in 
Figure  3-2  was  generated  by  simulating  RSS  evaluations  many  times  while,  in 
each  instance,  randomly  choosing  input  factors  from  preestablished  “reasonable” 
statistical  distributions  for  each  of  the  evaluation  factors.  (Appendix  B  documents 
the  assumed  distributions  for  the  evaluation  factors.)  Based  on  the  central  limit 
theorem,  one  would  expect  an  approximately  normal  distribution  of  the  resulting 
evaluations,  even  if  distributions  of  individual  evaluation  factors  are  not  normal 
and  not  identically  distributed.6  The  approximately  normal  distribution  is  readily 
apparent  in  Figure  3-2. 
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Figure  3-2.  Simulated  Distribution  of  Many  Evaluations 


We  made  use  of  this  simulated  distribution  to  define  relative  risk  in  terms  of  the 
distribution’s  calculated  mean  and  standard  deviation.  The  general  idea  is  illus¬ 
trated  in  Figure  3-3. 

Figure  3-3.  Example  Assessment  of  Risk 
Sector  A  Sector  B 
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In  Figure  3-3,  the  area  between  -la  (standard  deviation)  and  +  la  is  arbitrarily 
colored  green  to  indicate  the  range  of  acceptable  risk.  Together,  this  and  the  other 
areas  on  this  figure  mean  the  following: 


Range 

Color 

Meaning 

-4g  to  -2a 

Red 

Unacceptable  risk 

-2a  to  -1  a 

Yellow 

Questionable  risk 

-la  to  +1a 

Green 

Acceptable  risk 

+1a  to  +2c 

Blue 

Low  risk 

+2a  to  +4  a 

Light  blue 

Very  low  risk 

With  this  backdrop,  if  the  Sector  A  score  of  0.312  and  the  Sector  B  score  of  0.321 
plotted  in  the  locations  indicated,  then  one  could  conclude  that  the  risk  of 
Sector  A  was  acceptable  (but  close  to  the  questionable  region),  while  Sector  B  had 
low  risk.  (Similar  comparisons  can  be  performed  on  each  of  the  individual  cus¬ 
tomer  needs.) 

The  color-rating  process  just  described  is  consistent  with  that  used  in  DoD  pro¬ 
curement  source-selection  procedures.  For  that  matter,  so  is  the  method  of  estab¬ 
lishing  criteria,  weighting  the  criteria,  and  evaluating  alternatives  against  criteria.7 
However,  in  source  selections  the  dividing  points  between  acceptable  and  unac¬ 
ceptable  risk  are  entirely  judgmental.  Thus,  one  of  the  things  we  have  added  to  the 
usual  practice  is  a  statistically-based  rule  for  establishing  those  breakpoints.  Even 
more  important,  though,  is  that  if  the  basic  procedure  is  acceptable  for  procure¬ 
ments  generally,  then  it  logically  also  ought  to  be  acceptable  for  what  amounts  to 
a  close  companion — the  repair  sector  selection  decision. 
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Chapter  4 

Testing  the  New  Model  and  Process 
Results  and  Recommendations 


Repair  sector  decisions  are  by  nature  controversial,  and  there  is  a  history  of  previ¬ 
ous  approaches  to  the  repair  sector  decision  that  were  partially  successful,  were 
only  partially  implemented,  or  that  simply  were  not  adequate  to  the  task.1  Thus  it 
is  important  to  demonstrate  that  the  model  described  in  Chapter  2  and  the  decision 
process  described  in  Chapter  3  work  in  practice. 

We  tested  the  new  model  and  process  on  three  different  V-22  aircraft  workloads. 
This  chapter  describes  the  conduct  of  and  results  of  the  trial  applications. 

Process  for  Conducting  Trial  Applications 

We  implemented  the  three-tier  evaluation  model  described  in  Chapter  2  in  a  soft¬ 
ware  product  called  QFD/Capture  and  in  an  Excel  spreadsheet.  QFD/Capture, 
because  of  its  graphical  nature,  was  a  better  tool  for  establishing  the  relative  im¬ 
portance  of  repair  customer  needs,  assigning  the  weights  that  link  customer  needs 
to  the  characteristics  of  choices,  and  assigning  the  weights  linking  characteristics 
of  choices  to  determining  factors.  The  Excel  spreadsheet  was  used  to  do  calcula¬ 
tions  because  Excel  offered  certain  calculation  capabilities  not  available  in 
QFD/Capture. 

The  importance  of  each  customer  need  was  initially  assigned  by  LMI  analysts  and 
then  checked  for  reasonableness  during  field  interviews.  We  also  developed  a 
preliminary  set  of  weights  linking  customer  needs  to  the  characteristics  of  choices 
and  characteristics  of  choices  to  determining  factors.  We  developed  the  weights 
using  a  focus  group  comprising  LMI  staff  members  knowledgeable  in  depot 
maintenance,  acquisition,  sustaining  engineering,  inventory  management,  and 
modem  manufacturing  methods.  Every  effort  was  made  to  establish  reasonable 
weights,  and  the  final  weights  include  feedback  from  trial  evaluations.  However, 
the  weights  largely  remain  the  product  of  this  one  focus  group.  Future  corrobora¬ 
tion  by  another  focus  group  or  by  other  means  is  appropriate. 

Using  the  evaluation  model  described  in  Chapter  2  (with  LMI-developed  weights) 
and  the  decision  process  described  in  Chapter  3,  the  V-22  system  program  office 
(SPO)  conducted  trial  evaluations  on  the  T-406  engine,  the  V-22  forward-looking 
infrared  scanner,  and  the  Cockpit  Control  Feel  and  Drive  actuator.  These  three 
subsystems  were  legitimate  repair  sector  decision  candidates  that  the  V-22  SPO 
was  actively  evaluating.  They  are  also  representative  of  the  span  of  components 
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that  might  be  subject  to  evaluation,  since  one  of  them  embodies  propulsion  tech¬ 
nology,  one  is  an  avionics  subsystem,  and  one  is  electromechanical. 

For  each  of  the  components,  evaluators  assessed  alternative  repair  sectors  using 
the  factors  described  in  Chapter  2.  The  overall  process  is  illustrated  in  Figure  4-1. 
Attendees  received  evaluation  workbooks  one  or  more  days  before  the  evaluation 
session  so  that  they  could  do  preparatory  research.  The  sessions  themselves  began 
with  an  opportunity  approximately  1  hour  in  length  for  government  and  contractor 
representatives  to  give  an  overview  briefing  (this  opportunity  was  not  always 
used)  or  make  introductory  remarks.  The  evaluations  themselves  took  about 
4  hours.  The  times  shown  in  the  “Analysis”  column  in  Figure  4-1  are  representa¬ 
tive  of  the  effort  required  for  one  analyst  to  perform  these  tasks. 

Figure  4-1.  Process  for  Conducting  Trial  Applications 


Prior  to  evaluation 


Workbooks 
provided  to 
participants 
one  to  two 
days  in 
advance 


Joint  government/ 
industry  evaluation 


Overview  of 
process 


30  minutes 


Contractor/ 
government  briefings 
and  remarks 


Joint  evaluation  of 
factors 


4  hours 


Analysis 


Preliminary 

results 


1  hour 


Error  check, 
sensitivity 
analysis 

1 .5  hours 


Outbrief 
(prep  and 
present) 


3  hours 


The  four  evaluations  that  took  place  are  summarized  in  Table  4-1.  LMI  staff 
members  who  had  contributed  to  the  methodology  acted  as  facilitators  for  the 
evaluation  sessions  but  did  not  contribute  to  the  assignment  of  scores.  The  gov¬ 
ernment  and  contractor  representatives  worked  as  an  integrated  process  team  to 
reach  consensus  on  both  the  organic  and  contract  alternative  scores.  In  the  few 
cases  where  it  became  apparent  that  the  consensus  was  becoming  forced,  our 
facilitators  kept  track  of  the  range  of  scores  and  used  the  ranges  later  for  sensitiv¬ 
ity  analysis.  After  the  evaluators  scored  the  repair  sectors  on  each  factor,  we  com¬ 
piled  the  results,  did  the  sensitivity  analysis,  and  conducted  the  outbriefing. 
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Testing  the  New  Model  and  Process — Results  and  Recommendations 


Table  4-1.  Trial  Evaluations 


Session 

number 

Component 

evaluated 

Panel  membership 

1 

T-406  engine 

System  program  office  and  cognizant  mainte¬ 
nance  depot 

II 

T-406  engine 

System  program  office,  cognizant  maintenance 
depot,  and  engine  OEM 

III 

FUR 

System  program  office,  cognizant  maintenance 
depot,  and  FUR  OEM 

IV 

CCFD 

System  program  office,  cognizant  maintenance 
depot,  and  airframe  OEM  (did  not  include  CCFD 
vendor) 

Trial  Application  Results 

We  expected  that  a  sequence  of  trials  would  be  required  to  refine  the  evaluation 
scales  and  process,  and  this  proved  to  be  the  case.  The  major  problems  we  en¬ 
countered  are  summarized  in  Table  4-2.  The  first  problem  listed  had  to  do  with 
the  model  itself,  and  the  next  four  involved  how  the  evaluation  was  conducted. 
The  final  problem,  as  indicated,  may  or  may  not  actually  be  a  problem  and  will 
require  further  trial  evaluations  to  better  characterize.  We  discuss  each  of  these 
problems  below. 

New  Factors 

Evaluators  sometimes  were  unable  to  find  a  factor  that  would  let  them  express  a 
difference  between  the  repair  sectors  in  an  area  that  they  thought  was  important. 
We  identified  the  missing  factors  and,  after  additional  research  confirmed  their 
relevance,  added  them.  Examples  are  reserve  capacity  for  the  proposed  workload, 
capacity  in  excess  of  workload,  customer  knowledge;  and  policy  on  repair  parts. 

Scales  with  Vague  Anchors  or  Confusing  Terminology 

In  some  cases  the  evaluators  agreed  with  the  need  for  the  factor  they  were  evalu¬ 
ating  but  had  difficulty  agreeing  on  its  meaning;  it  would  take  extensive  conver¬ 
sation  to  define  it,  and  later  they  would  have  difficulty  remembering  the 
definition.  This  confusion  could  lead  to  possible  inconsistency  bias.  The  key  to 
resolving  this  problem  was  better-defined  anchors.  For  example,  Figure  4-2  shows 
the  initial  vague  set  of  anchors  for  the  factor  “demand  predictability”  and  the  final, 
more  concrete  anchors.  We  proceeded  to  refine  the  anchors  throughout  the  trials. 
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Table  4-2.  Problems  Noted  During  Trial  Evaluations 


Problem 

Approach  to  resolution 

Results 

Apparently  relevant  factors 
not  included. 

Added  factors. 

Problem  appeared  to  be 
resolved. 

Scales  for  some  factors  had 
vague  anchors  or  confusing 
terminology. 

Redefined  anchors  and  terminol¬ 
ogy. 

Problem  appeared  to  be 
resolved. 

Evaluators  did  not  understand 
the  relevance  of  some  of  the 
factors. 

For  each  factor  in  the  workbook, 
provided  a  graphical  display  indi¬ 
cating  which  repair-sector  char¬ 
acteristics  the  factor  affected. 

Problem  appeared  to  be 
resolved. 

Evaluators  did  not  have  suffi¬ 
cient  knowledge  of  organic 
and  contractor  capabilities 
and/or  proposed  business 
arrangements  to  fairly  evalu¬ 
ate  both  organic  and  con¬ 
tractor  alternatives. 

Included  contractor  representa¬ 
tives  in  Sessions  II,  III,  and  IV. 
(However,  this  problem  reoccurred 
in  Session  III,  when  the  govern¬ 
ment  evaluator  did  not  fully  under¬ 
stand  the  organic  capabilities.) 

The  process  as  designed  will 
not  adequately  resolve  this 
problem,  since  it  does  not  con¬ 
trol  who  will  be  selected  as 
evaluators. 

Evaluators  had  a  stake  in  out¬ 
come,  leading  to  apparent 
(though  possibly  unintended) 
motivational  bias. 

Included  experts  from  both 
government  and  industry  repair 
sectors. 

The  process  as  designed  will 
not  adequately  resolve  this 
problem,  since  it  does  not  con¬ 
trol  who  will  be  selected  as 
evaluators. 

Preponderance  of  low-risk 
evaluations  appeared  in  trial 
evaluations. 

Not  clear  whether  this  reflects  re¬ 
ality  (i.e.,  the  repair  sectors  were 
actually  low  risk  for  the  subsys¬ 
tems  evaluated)  or  is  an  artifact  of 
the  risk  analysis  procedure.  Addi¬ 
tional  evaluations  are  needed  to 
better  characterize. 

Not  fully  resolved,  because  the 
cause  is  not  certain. 

Relevance  of  Factors  Not  Clear 

Evaluators  sometimes  did  not  understand  why  they  were  evaluating  a  particular 
factor  and  what  repair  sector  characteristics  a  factor  affected.  We  addressed  this 
problem  by  revising  the  workbook  to  include  a  display  like  that  shown  in  Figure 
4-3,  for  the  factor  “reserve  capacity.”  The  filled  dots  in  this  display  indicate  that 
the  factor  affects  adequate  capability  and  capacity,  reprioritization  capability,  and 
surge  capability.  The  fact  that  the  dots  are  completely  filled  (as  opposed  to  three- 
quarters  filled,  half  filled,  quarter  filled,  or  open)  indicates  that  the  relationship  of 
reserve  capacity  to  the  characteristics  is  very  important. 
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Figure  4-2.  Revised  Factor  Scale  Anchors 


1 1 1-3.  Demand  predictability 
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predictability 
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predictability 
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predictability 

Revised 

anchors 

Forecasting  of  little 
or  no  avail.  Peaks 
and  valleys  of 
workload  are  so 
extreme  that  best 
plan  is  to  react. 

Forecasting  of 
some  use. 
However, 
workload  will  vary 
from  calendar 
quarter  to 
calendar  quarter. 

Not  deterministic, 
has  peaks  and 
valleys,  but  amount 
of  workload 
consistent  from  one 
calendar  quarter  to 
another. 

Deterministic. 
No  surprises. 

Forecast 
workload  and 
actual  workload 
are  identical 
(e.g.,  time 
change  items) 

Figure  4-3.  Workbook  Display  Establishing  Relevance  of  Factor 


Evaluation  Factor:  V-4. 

Reserve  capacity  at  a  potential  source  of  repair  for  this  work. 


This  factor  affects: 
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of  product 
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Insufficient  Knowledge  of  Capabilities  or  Proposed 
Business  Arrangements 

On  completion  of  Session  I,  which  did  not  include  evaluators  from  industry,  we 
suspected  that  a  number  of  biases  could  have  been  present.2  Because  of  the  possi¬ 
bility  of  a  biased  result,  we  arranged  to  repeat  the  evaluation  with  evaluators  from 
the  engine  OEM  present  (Session  II).  The  differences  between  the  two  sessions 
were  striking. 
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In  Session  I,  as  an  example,  government  evaluators  were  convinced  that  the  en¬ 
gine  OEM  had  labor  difficulties  that  would  result  in  a  strike  within  weeks  of  the 
evaluation.  Discussion  during  Session  II  showed  this  belief  to  be  unfounded.  This 
erroneous  information,  however,  influenced  the  results,  since  labor  relations  is  an 
evaluation  factor.  The  problem,  of  course,  is  that  near-term  labor  issues  sur¬ 
rounding  one  contractor  are  largely  irrelevant  in  the  context  of  a  long-term  repair 
sector  decision.  This  is  a  clear  case  of  availability  bias.  Similarly,  the  “power-by- 
the-hour”  business  arrangement  that  had  been  proposed  by  the  engine  OEM  was 
not  clearly  understood  by  some  of  the  government  representatives,  and  that  lack  of 
understanding  affected  most  of  the  factors  under  the  category  of  nature  of  rela¬ 
tionship  with  sources  (probable  misinterpretation  bias).  It  also  turned  out  that  the 
engine  OEM  did  not  have  a  complete  picture  of  organic  capabilities. 

The  difference  that  contractor  representation  made  is  quite  evident  in  the  overall 
evaluation  scores.  Figure  4-4  contrasts  the  overall  results  of  Sessions  I  and  II. 

Both  organic  and  contract  scores  improved;  the  contract  score  improved  dramati¬ 
cally,  resulting  in  a  reversal  of  the  source  preference  from  organic  to  contractor. 

Figure  4-4.  Comparing  Results  of  Sessions  I  and  II 


Session  I  Session  II 


High  risk  < -  Overall  risk  - ►  Low  risk 


It  should  be  emphasized  that  in  Session  II  the  combined  evaluation  panel  (SPO 
personnel,  organic  depot  personnel,  and  contractor  personnel)  generated,  for  the 
most  part,  consensus  evaluations  of  both  the  organic  and  contractor  alternatives  as 
an  integrated  team.  That  is,  it  was  not  a  case  of  the  contractor  scoring  the  con¬ 
tractor  alternative  and  the  government  scoring  the  organic  alternative.  (As  noted 
earlier,  in  the  few  instances  where  it  appeared  that  consensus  would  be  forced,  the 
LMI  facilitators  kept  track  of  the  range  of  scores  and  used  the  ranges  later  in  sen¬ 
sitivity  analysis.) 
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Session  III  had  almost  the  opposite  of  the  problem  encountered  in  Session  I,  be¬ 
cause  the  individual  available  to  evaluate  for  the  government  did  not  have  a  solid 
grasp  of  organic  capabilities  or  issues.  (We  did  not  have  the  opportunity  to  repeat 
the  FLIR  evaluation.) 

Evaluators  with  Stake  in  Outcome 

Another  way  to  read  the  difference  between  Sessions  I  and  II  is  that  government 
evaluators  in  Session  I,  who  were  from  a  government  depot,  had  a  stake  in  the 
outcome  and  thus  a  conflict  of  interests.  In  the  trial  applications  of  the  decision 
process,  we  addressed  this  potential  problem  by  bringing  representatives  of  both 
industry  and  government  depots  to  the  table  as  an  IPT.  As  will  be  discussed  later 
in  this  chapter,  we  do  not  believe  this  approach  is  an  adequate  safeguard  against 
the  motivational  biases  that  can  arise  from  conflicted  evaluators. 

Preponderance  of  Low-Risk  Evaluations 

In  Figure  4-4,  above,  both  the  government  and  contractor  alternatives  in  Session  II 
were  regarded  as  having  low  risk.  In  this  particular  case  our  holistic  sense  of  the 
alternative  repair  sectors  suggests  such  a  result  is  not  unrealistic.  However,  this 
same  pattern,  assessment  of  low  risk  for  both  alternatives,  prevailed  in  Sessions  III 
and  IV  as  well.  We  are  not  sure  whether  the  results  are  an  artifact  of  the  limited 
sample  we  examined  (i.e.,  they  were  simply  all  low  risk)  or  of  the  evaluation  pro¬ 
cedure  itself.  Only  additional  evaluations  can  shed  light  on  this  question. 

Assessing  the  Repair  Sector  Decision  Model 
and  Decision  Process 

In  this  section  we  present  an  overall  evaluation  of  both  the  repair  sector  decision 
model  and  the  repair  sector  decision  process,  beginning  with  discussion  of  the 
model. 

Usefulness  of  the  Model 

In  Chapter  2,  we  identified  27  potential  evaluation  factors  in  the  three  categories 
of  nature  of  repair  work  itself,  nature  of  relationship  with  sources,  and  nature  of 
the  sources  of  repair.  There  was  support  for  the  model  elements  at  all  three  levels, 
and  the  evaluation  sessions  indicated  the  overall  conceptual  model  is  reasonable; 
but  it  still  remained  to  be  determined  which  of  the  27  evaluation  factors  should  be 
retained.  Reasonable  criteria  for  retaining  a  factor  should  include  the  following, 
for  the  reasons  indicated: 

♦  The  factor  is  important  to  satisfying  customer  needs.  Since  it  requires  ef¬ 
fort  to  obtain  data  on  any  of  the  factors,  a  factor  that  is  not  important 
should  be  discarded  in  the  interest  of  a  parsimonious  model. 
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♦  The  factor  evaluation  is  sensitive  to  the  choice  of  repair  sector  being 
evaluated.  Even  if  a  factor  is  important  to  a  customer,  if  the  choice  of  re¬ 
pair  sector  does  not  improve  or  impair  ability  to  satisfy  the  customer’s 
needs  then  the  factor  is  not  relevant  to  the  repair  sector  decision. 

♦  The  construct  behind  the  factor  is  understood  by  the  evaluators.  If  it  is  not, 
the  factor  is  not  meaningfully  contributing  to  the  decision  and  may  instead 
be  introducing  random  biases. 

♦  Evaluators  are  able  to  obtain  sufficient  information  on  the  factor  to  evalu¬ 
ate  it. 

We  concluded  that  22  of  the  original  27  factors  met  all  of  the  criteria.  Appendix  D 
summarizes  the  evaluation  results  for  each  of  the  original  27  factors.  The  five 
factors  that  should  probably  be  eliminated  and  the  main  reasons  for  eliminating 
them  are  summarized  in  Table  4-3. 

Table  4-3.  Factors  That  Are  Candidates  for  Elimination 


Factor 

Rationale  for  eliminating 

Workload  size  versus  size 
of  market 

Relative  size  of  workload  compared  to  marketplace  for 
similar  work  is  not  important  to  choice  of  repair  sector.  The 
more  relevant  factor  was  size  of  workload  compared  to 
each  repair  sector. 

Technology  renewal  rate 

Not  helpful  in  choosing  between  repair  sectors,  since  pref¬ 
erence  for  one  sector  or  the  other  does  not  depend  on  un¬ 
derlying  technology  renewal  rate. 

Uniqueness  of  technology 

Not  helpful  in  choosing  between  repair  sectors,  since  pref¬ 
erence  for  a  sector  does  not  depend  on  uniqueness  of 
technology. 

Availability  of  engineering 

Not  relevant  to  the  choice  of  repair  sector.  The  relevant 
factor  is  absence  of  proprietary  data. 

Overhead  structure 

Evaluators  had  difficulty  evaluating  this  factor,  especially 
when  attempting  to  generalize  over  all  possible  commercial 
sources  of  repair  or  over  all  possible  organic  sources  of 
repair. 

The  retained  factors,  along  with  their  ordinal  ranks  (i.e.,  rank  based  on  computed 
importance)  prior  to  eliminating  any  factors,  are  listed  in  Table  4-4. 

Perhaps  it  is  not  surprising  that  the  factors  most  important  to  getting  a  good  deal 
for  the  repair  services  customer  were  incentive  structure  and  customer  knowledge, 
followed  closely  by  type  of  relationship  and  availability  of  alternative  sources. 

At  least  two  of  the  22  retained  factors,  however — reserve  capacity  for  this  work 
and  capacity  in  excess  of  workload — will  require  work  beyond  that  accomplished 
in  this  study.  We  did  not  completely  resolve  problems  associated  with  assessing 
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capacity;  evaluators  had  difficulty  with  the  capacity  construct  in  each  evaluation 
session. 


Table  4-4.  Retained  Evaluation  Factors 


Category 

Decision  factor  and  rank 

Nature  of  the  repair  work 

•  Relative  size  of  work  compared  to  work  size  of 
source  of  repair  (13) 

•  Demand  predictability  (24) 

•  Design  stability  (23) 

•  Repair  process  dependence  (17) 

•  Absence  of  proprietary  data  issues  (1 8) 

Nature  of  relationship  with 
sources 

•  Type  of  relationship  (flexible  vs.  inflexible)  (3) 

•  Scope  of  relationship  (15) 

•  Policy  on  related  functions  (9) 

•  Policy  on  lot  sizes  (14) 

•  Policy  on  repair  parts  (7) 

•  Incentive  structure  (1) 

Nature  of  sources 

•  Availability  of  more  than  one  source  (4) 

•  Availability  of  sources  with  complementary  workload 
(12) 

•  Interest  by  potential  sources  (8) 

•  Reserve  capacity  for  this  work  (21 ) 

•  Capacity  in  excess  of  workload  (19) 

•  Availability  of  skilled  labor  (10) 

•  Stability  of  labor  relationships  (27) 

•  Customer  knowledge  (2) 

•  Product  knowledge  (11) 

•  Repair  process  knowledge  (6) 

•  MRP-II  consistent  management  methods  (5) 

Evaluation  Process 

The  process  poses  two  potential  issues.  The  first  is  how  much  effort  the  process 
takes.  The  second  is  what  works  and  what  does  not.  We  begin  by  discussing  how 
much  effort  is  required  to  operate  the  decision  process. 

Effort  Required  to  do  an  Evaluation 

The  overall  evaluation  process  was  illustrated  earlier  in  Figure  4-1.  Sessions 
lasted  about  5  hours,  including  both  introductory  presentations  and  the 
evaluations  per  se.  Of  that  total,  the  evaluations  themselves  took  about  4  hours. 
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As  indicated  in  Figure  4-1,  the  time  required  for  post-evaluation  analysis  was 
fairly  modest. 

Since  the  evaluations  in  our  design  depend  on  the  availability  of  workbooks,  re¬ 
lated  questions  are  how  many  different  workbooks  might  be  needed,  and  how  long 
it  will  take  to  prepare  one.  The  best  answers  will  come  from  experience,  but  these 
are  reasonable  extrapolations  from  the  trial  application: 

♦  One  workbook  will  be  needed  for  each  major  category  of  workloads.  The 
major  categories  of  workloads  will  probably  comprise  propulsion  (covered 
by  the  workbook  most  completely  developed  as  part  of  this  study),  inte¬ 
grated  electronics,  simple  electronics,  structures,  and  mechanical  compo¬ 
nents. 

♦  Since  additional  workbooks  can  be  developed  by  modifying  the  propulsion 
example,  developing  a  single  workbook  should  not  take  more  than  3  to  5 
days  of  effort  on  the  part  of  two  or  three  competent  maintenance  analysts. 

A  4-hour  evaluation  proceeded  by  1  hour  of  briefings  and  some  prior  preparation 
does  not  seem  exorbitant  for  a  major  subsystem.  Our  expectation  is  that  the  total 
repair  sector  selection  decision  time  and  resource  commitments,  using  the  model 
and  process  described  in  this  report,  probably  are  less  intrusive  than  has  histori¬ 
cally  been  the  case.  The  reason  is  that  more  of  the  problem  is  addressed  at  one  sit¬ 
ting,  reducing  the  need  for  extended  iteration  as  new  questions,  not  previously 
considered,  are  submitted  by  interested  parties. 

Overall  Process  Effectiveness 

Generally  the  evaluation  process  appeared  to  provide  a  reasonably  objective  way 
of  looking  at  the  key  factors  that  determine  a  repair  sector’s  ability  to  meet  cus¬ 
tomer  needs.  That  is,  it  was  a  reasonably  comprehensive  model  of  the  key  ele¬ 
ments  of  the  public  versus  private  repair  sector  selection  decision. 

Further,  the  evaluators  found  it  of  benefit.  By  addressing  notionally  all  of  the  rele¬ 
vant  considerations — with  factors  and  measurement  scales  defined — it  enabled 
them  to  talk  to  each  other  about  the  important  issues,  rather  than  past  each  other. 
As  they  perceived  it,  the  structured,  joint  assessment  of  each  other’s  capabilities 
by  industry  and  government  yielded  a  better  joint  understanding  of  depot-level 
repair  issues  and  a  better  understanding  of  each  other’s  strengths  and  limitations. 

However,  we  are  not  convinced  that  our  process  by  itself  will  assure  fair  or  con¬ 
sistent  evaluations  across  the  military  services  and  over  time.  Although  it  miti¬ 
gated  some  sources  of  bias,  others  remain  or  are  likely  to  recur,  depending  on  the 
particular  individuals  who  participate  in  the  evaluation  focus  group. 
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Consistent  and  fair  evaluations  would  logically  depend  on  these  ingredients: 

♦  A  reasonably  comprehensive,  fair,  and  consistent  way  to  look  at  the  repair 
sector  selection  decision. 

♦  Evaluators  who  have  training  and  experience  to  do  evaluations  well.  This 
training  and  experience  would  necessarily  cover  the  relevant  technical  as¬ 
pects  of  maintenance  and  business  practices  (management  and  financial)  in 
both  the  commercial  and  government  worlds.  It  would  also  necessarily  in¬ 
clude  the  repair  sector  selection  model  and  the  procedures  for  evaluating 
the  input  factors. 

♦  Evaluators  who  are — and  are  perceived  as  being — free  of  a  personal  or 
organizational  stake  in  the  outcome. 

The  experience  with  the  trial  evaluations  (as  well  as  the  history  of  decision  tree 
analysis)  suggests  that  the  second  and  third  criteria  are  not  satisfied.  Although 
practices  differ  across  services,  initial  RSS  decisions  are,  today,  typically  made  by 
the  staffs  of  system  program  offices.  These  individuals  generally  lack  prior  expe¬ 
rience  in  such  decisions  and/or  related  training.  (With  the  exception  of  senior 
staff,  this  was  the  case  in  the  trial  applications.)  Further,  once  they  make  the  in¬ 
vestment  in  mastering  the  process  they  may  not  have  the  opportunity  to  participate 
in  such  decisions  again. 

There  certainly  is  no  cadre  of  personnel  with  an  understanding  of  both  industry 
and  government  capabilities.  Traditional  decision  tree  analyses  have  made  the 
choice  between  organic  and  contract  repair  with  little  if  any  private-sector  partici¬ 
pation.  Inevitably,  as  our  experiments  tend  to  confirm,  characterizing  industry 
without  really  understanding  its  capabilities  and  limitations  results  in  biased 
evaluations  to  the  detriment  of  the  final  customer. 

Although  the  evaluations  described  in  this  report  were  putatively  between  the  or¬ 
ganic  and  contract  repair  sector  alternatives  rather  than  between  specific  organic 
and  commercial  sources,  for  two  of  the  subsystems  it  was  reasonably  clear  who 
the  real  stakeholders  were  on  both  the  organic  and  contract  sides.  We  were  fortu¬ 
nate  to  obtain  the  cooperation  of  industry  stakeholders  for  evaluations  in  Sessions 
II  and  III  and  the  organic  stakeholders  for  all  four  (with  limitations  in  Session  III). 
The  problem  is  that  there  will  be  situations  where  specific  private-sector  repre¬ 
sentation  is  not  practical.  An  example  is  where  there  are  multiple  potential  com¬ 
mercial  sources  of  repair.  Similarly,  the  experience  of  Session  III  suggests  that 
adequate  knowledge  of  government  capability  cannot  be  assured. 

Even  where  there  is  prior  experience,  the  provision  regarding  a  lack  of  conflict 
cannot  be  satisfied.  For  instance,  the  government  personnel  we  interviewed  or 
worked  with  during  this  project  generally  had  an  organizational  stake  in  the  RSS 
decision.  The  resulting  problems  were  readily  apparent  during  the  conduct  of  the 
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four  trial  evaluations,  particularly  Session  I:  when  one  side  or  another  comes  into 
a  session  making  it  clear  from  the  outset  that  only  one  answer  will  satisfy  them, 
then  bias  is  an  inevitable  result.  A  strong  and  fair  government  moderator  miti¬ 
gated  the  problem  during  Sessions  II  and  III.  The  problems,  however  are  that 

♦  a  strong  and  fair  moderator  might  not  always  be  available  and 

♦  industry  personnel  and  government  personnel  may  not  be  peers  in  terms  of 
education  and  technical  training.  When  they  are  not  then  one  set  of  repre¬ 
sentatives  can  outclass  the  other. 

Additional  Necessary  Process  Changes 

To  resolve  the  remaining  important  problems  will  require  a  more  robust  approach 
than  an  IPT  or  focus  group.  An  approach  that  commends  itself  is  the  due  diligence 
procedure  that  businesses  use  to  study,  investigate,  and  evaluate  major  business 
opportunities.  The  purposes  of  a  due  diligence  procedure  are  to 

♦  determine  (generally  through  interviews,  document  study,  and  on-site  in¬ 
spection)  that  a  business  is  what  it  seems  or  is  represented  to  be; 

♦  verify  that  a  proposed  investment  complies  with  investor’s  criteria;  and 

♦  provide  a  defense  against  third-party  claims  at  some  later  time. 

The  RSS  decision  involves  very  similar  considerations — that  the  commercial  and 
organic  sectors  are  as  they  appear  to  be,  that  one  or  the  other  will  better  fit  DoD’s 
criteria  for  supporting  its  repair  customers,  and  to  document  that  the  RSS  decision 
was  fair,  impartial,  and  in  the  best  interest  of  the  repair  customer. 

Of  the  characteristics  of  the  ideal  due  diligence  investigator  or  investigation  team, 
two  are  particularly  relevant  here.  First,  due  diligence  investigators  normally  are 
sufficiently  senior  that  they  grasp  corporate  politics,  power  structures,  cultures, 
and  similar  considerations.  An  investigator  who  lacks  this  background  might  pro¬ 
duce  a  mechanically  satisfactory  analysis  but  be  unable  to  provide  the  broad  in¬ 
sight  needed  for  a  really  sound  decision.  Second,  due  diligence  investigators  are 
normally  unaffected  by  the  outcome  of  the  due  diligence  process.  This  provision 
is  necessary  to  avoid  conflicts  of  interest  and  help  assure  objectivity.  It  is  particu¬ 
larly  this  second  provision  that  we  call  to  attention  here.  As  discussed  above,  the 
RSS  evaluations  that  are  made  today  are  not  accomplished  by  individuals  without 
a  stake  in  the  outcome. 
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Testing  the  New  Model  and  Process — Results  and  Recommendations 


Accordingly,  we  recommend  that  DoD  borrow  from  commercial  due  diligence 
practice  and  create  an  RSS  decision  support  cadre  reporting  to  the  Defense  Depot 
Maintenance  Council.  This  cadre  should  be  structured  to  be 

♦  truly  without  conflict  of  interest  regarding  the  outcome  of  RSS  decisions 
and  able  to  maintain  the  confidence  of  the  competing  constituencies  as 
well  as  of  internal  and  external  process  or  decision  critics;  and 

♦  sufficiently  senior  and  experienced  to  bring  to  the  RSS  analyses  the  requi¬ 
site  education,  technical  training,  market  awareness,  and  policy  perspec¬ 
tives.  In  particular,  there  should  be  no  question  but  that  they  are  on  a  peer 
level  with  their  organic  depot  and  commercial  industry  counterparts. 
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End  Notes  for  Chapter  4 


1  To  illustrate,  in  1982,  the  Office  of  the  Secretary  of  Defense  directed  the  services  to  use  a  decision 
tree  analysis  process  for  repair  sector  decisions.  A  1990  LMI  study  of  Army  repair  sector  decisions 
could  not  find  evidence  that  it  had  been  used  or  disseminated  in  the  Army.  Kelvin  K.  Kiebler,  Larry 
S.  Klapper,  and  Donald  T.  Frank,  Army  Depot  Maintenance:  More  Effective  Use  of  Organic  and 
Contractor  Resources ,  LMI  AR803R1,  June  1990,  p.  2-20; 

The  1994  Defense  Science  Board  study  stated  that  the  services  had  and  used  decision  tree  analyses 
but  that  the  processes  were  inconsistent  with  the  CORE  concept.  Office  of  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology,  Report  of  the  Defense  Science  Board  Task  Force  on  De¬ 
pot  Maintenance  Management ,  April  1 994,  Appendix  E. 

2  These  were  selfishness  and  guile,  impression  management,  information  availability,  misinterpreta¬ 
tion,  and  failure  to  seek  evidence  to  the  contrary.  See  Table  3-1. 

3  Gordon  Ring,  Due  Diligence  Techniques  and  Analysis,  Westport,  CN:  Quorum  Books,  1996,  pp. 
2,  52. 
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Mathematical  Form  of  the  Model 


Introduction 

Chapter  2  of  this  report  described  a  conceptual  model  of  the  repair  sector  selection 
decision  that  included  three  sets  of  elements: 

♦  Needs  of  the  end-users  of  depot  repair  services  (e.g.,  timeliness,  low  cost, 
quality) 

♦  Characteristics  of  the  two  repair  sectors  relevant  to  satisfying  user  needs 
(e.g.,  adequate  capability  and  capacity,  scope  and  scale  economy,  respon¬ 
siveness) 

♦  Factors  that  determine  the  characteristics  of  the  repair  sectors.  Such  factors 
fall  into  three  categories,  according  to  whether  they 

►  are  intrinsic  to  the  nature  of  the  repair  sources  themselves, 

►  depend  on  the  relationship  between  the  repair  source  and  the  buyer  of 
repair  services,  or 

>-  are  a  function  of  the  repair  work  to  be  accomplished. 

The  conceptual  model  has  at  its  starting  point  the  determination  of  a  repair  sec¬ 
tor’ s  ability  to  satisfy  repair  customer  needs.  However,  because  of  the  many  fac¬ 
tors  that  come  into  play,  it  is  too  simplistic  to  think  of  that  ability  as  a  yes-or-no 
proposition.  Rather,  what  we  are  interested  in  is  the  degree  to  which  one  repair 
sector  will  probably  satisfy  customer  needs  when  compared  to  another,  i.e.,  the 
relative  risk  of  each  sector.  To  establish  that  each  of  the  needs,  characteristics,  and 
factors  is  a  legitimate  element  in  determining  repair  sector  risk,  Chapter  2  pre¬ 
sented  support  from  previous  depot  maintenance  studies,  from  the  relevant  gen¬ 
eral  literature,  or  both.  To  test  the  conceptual  model  we  implemented  it  in  a 
mathematical  form  and  then  used  that  to  run  trial  applications  on  three  different 
V-22  aircraft  workloads.  This  appendix  describes  the  mathematical  form  of  the 
model. 

Prior  to  describing  the  source  of  model,  we  first  describe  the  alternative  ap¬ 
proaches  considered  and  the  one  that  forms  the  basis  for  the  model.  We  then  show 
how  that  approach  was  implemented  in  the  repair  sector  selection  decision  model. 
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Mathematical  Modeling  Approach 


Five  primary  considerations  were  important  to  the  choice  of  mathematical  form: 

1 .  The  mathematical  approach  needed  to  accommodate  economic  factors 
such  as  capacity  in  excess  of  workload  and  noneconomic  factors  such  as 
customer  knowledge. 

2.  Most  of  the  factors,  both  economic  and  noneconomic,  were  more  qualita¬ 
tive  in  nature  than  quantitative — thus  a  mechanism  for  converting  from 
qualitative  (verbal)  to  quantitative  judgments  was  needed. 

3.  During  acquisition  of  new  weapon  systems  and  modifications,  when  this 
methodology  is  to  be  applied,  even  notionally  quantitative  data  are  notori¬ 
ously  soft.  Thus  the  mathematical  approach  needed  to  accommodate  un¬ 
certainty. 

4.  For  a  complex  weapon  system  many  repair  sector  selection  decisions  will 
be  made.  This  means  that  for  any  single  repair  sector  selection  decision  the 
effort  needed  to  gather  input  data  has  to  be  modest. 

5.  In  practice,  the  analysis  work  behind  repair  sector  selection  decisions  is 
decentralized  to  individual  program  offices.  Therefore,  it  was  important  to 
develop  a  mathematical  form  that  program  offices  could  use  without  help 
from  the  analysts  that  created  it. 

With  these  considerations  in  mind,  we  examined  alternative  mathematical  mod¬ 
eling  approaches  (summarized  in  Table  A-l). 

Simulation,  optimization,  and  expert  systems  are  relatively  familiar  modeling 
methods.  The  first  two  are  extensively  used  in  logistics.  Expert  systems  are  a  more 
recent  addition  to  the  analyst’s  tool  bag  but  are  now  well  covered  in  the  literature 
on  decision  support  systems1  and  have  documented  application  in  logistics.2 

Multiattribute  utility  theory  (MAUT)  is  a  technical  name  for  the  scoring  models 
often  used  to  select  among  policy  alternatives.  Typically  such  scoring  models 
identify  relevant  criteria,  assign  importance  weights  to  them,  judgmentally  score 
each  alternative  against  the  criteria,  then  multiply  weights  times  scores  and  add 
the  results  to  determine  the  preferred  alternative.  This  approach  was  used,  as  an 
example,  on  depot  maintenance  management  options  in  the  1993  Integrated 
Management  of  Department  of  Defense  Depot  Maintenance  Activities  study.3 
Contractor  source  selection  also  uses  this  approach  widely.4  In  decomposing  a 
problem  into  a  number  of  dimensions  that  are  considered  individually,  MAUT 
models  are  not  necessarily  better  or  more  accurate  than  a  global  assessment.  They 
do  have  the  advantage  of  being  informative  and  increasing  understanding  of  a 
problem  under  consideration.5 
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Mathematical  Form  of  the  Model 


Table  A-l.  Alternative  Mathematical  Modeling  Approaches 


Approach 

Advantages 

Disadvantages 

Simulation 

•  Easy  to  analyze  flows,  cycles,  and 
costs 

•  If  graphical,  aids  in  visualization  of 
material  flows 

•  Hard  or  impossible  to  capture  qualitative 
technical,  programmatic,  and  policy  fac¬ 
tors 

•  Requires  continuing  support  from  analyst 

Optimization 

•  Can  include  qualitative  and  subjec¬ 
tive  factors 

•  Can  be  hard  to  understand  and  explain 
(black  box  with  complicated  mathematics) 

•  Requires  continuing  support  from  analyst 

Expert  systems 

•  In  its  rule-based  form  automates 
decision  tree  logic 

•  Can  include  all  relevant  factors 

•  “Brittleness” — i.e.,  tendency  to  give  non¬ 
sense  answers  outside  narrow  domain  for 
which  designed;  probably  not  suitable  for 
this  broad  a  domain 

•  Requires  assistance  from  ana¬ 
lyst/knowledge  engineer  to  modify 

Multiattribute  utility 
theory  (MAUT) 

•  Can  include  all  relevant  factors 

•  Reasonably  easy  to  modify 

•  Easy  to  get  lost  in  details,  forget  what  the 
question  was 

•  Does  not  lend  itself  to  collaborative  think¬ 
ing 

Analytic  hierarchy 
process  (AHP) 

•  Can  include  all  relevant  factors 

•  Reasonably  easy  to  modify 

•  Provides  means  for  averaging 
judgment  errors 

•  Easy  to  get  lost  in  details 

•  Can  be  exhausting  to  do  the  many  paired 
comparisons  required 

•  Hard  to  use  collaboratively 

Quality  function 
deployment 

•  Graphical  nature  makes  it  easier  to 
keep  track  of  big  picture 

•  Lends  itself  to  collaborative  thinking 

•  Good  track  record  on  analogous 
problems 

•  Newer  in  the  United  States  and  less  well- 
known  than  other  methodologies 

•  Implementing  companies  have  viewed  its 
use  as  a  competitive  advantage,  limited 
the  number  of  published  results 

The  analytic  hierarchy  process  (AHP)  works  similarly,  in  that  it  decomposes  a 
problem  into  a  number  of  dimensions,  determines  criteria  weights  for  each  of  the 
dimensions,  and  evaluates  alternatives  against  criteria.6  A  prominent  difference  is 
that  it  aids  in  the  assignment  of  weights  and  evaluations,  by  comparing  the  criteria 
against  each  other  two  at  a  time  and  then  comparing  alternatives  against  criteria 
two  at  a  time.  This  pair-wise  comparison,  although  tedious,  can  help  compensate 
for  individual  errors  in  judgment  through  a  mathematical  procedure  for  essentially 
averaging  errors. 

The  newest  decision  support  method  we  considered  is  quality  function  deployment 
(QFD).7  Although  the  literature  on  QFD  tends  to  emphasize  its  application  rather 
than  its  mathematical  basis,  examination  of  the  inner  workings  shows  it  to  be 
based  on  multiattribute  utility  theory.  One  may  think  of  QFD  as  a  graphical  and 
strongly  customer-oriented  instance  of  MAUT. 
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The  advantages  and  disadvantages  shown  in  Table  A-l  are  in  part  a  compilation 
of  those  documented  in  the  literature.8  But  they  also  reflect  our  own  experience 
with  these  techniques  in  the  areas  of  reliability  and  maintainability  investments, 
logistics  technology  choices,  selection  of  management  strategies,  and  other  deci¬ 
sion  environments  relevant  to  the  repair  sector  choice. 

The  five  considerations  enumerated  above,  and  an  understanding  of  the  environ¬ 
ment  in  which  the  methodology  would  be  used,  informed  the  mathematical  ap¬ 
proach  chosen.  We  eliminated  simulation  as  the  primary  tool  because  of  its 
inability  to  handle  the  many  qualitative  factors.  Although  optimization  can  handle 
qualitative  factors  (after  conversion  to  quantitative  equivalents),  we  eliminated 
optimization  models  because  of  their  complexity  and  the  continuing  analyst  sup¬ 
port  they  would  need.  Expert  systems  are  not  a  viable  candidate,  in  our  estimation, 
because  the  problem  domain  is  broad  rather  than  narrow  and  expert  systems  do 
poorly  with  broad  problems. 

Our  choice  is  what  amounts  to  an  extension  of  the  quality  function  deployment 
methodology.  QFD’s  strong  customer  orientation  is  particularly  useful  for  repair 
sector  selection  decisions,  as  is  its  graphical  presentation.  We  extended  QFD  in 
three  ways: 

♦  First,  the  internal  arithmetic  in  commercially  available  QFD  software,  con¬ 
sistent  with  the  notion  of  quality  function  deployment  from  the  customer 
to  the  provider,  generally  determines  the  importance  of  proposed  factors 
that  enter  into  satisfying  customer  needs.  It  does  not  provide  a  means  for 
projecting  how  well  the  customer’s  needs  are  actually  met.  However,  a 
comparative  risk  assessment  requires  some  measure  of  how  well  customer 
needs  are  met.  In  this  implementation,  we  can  also  start  with  an  assess¬ 
ment  of  each  repair  sector  against  the  factors  and  then  estimate  how  well 
each  repair  sector  will  satisfy  each  of  the  customers’  needs. 

♦  Second,  in  common  with  MAUT  generally,  QFD  provides  unitless  relative 
ratings  (e.g.,  0.9,  2.6,  7.3).  A  relative  rating  of,  say,  2.63  for  one  repair 
sector  and  2.72  for  another  only  establishes  that  one  rating  is  higher  than 
the  other,  not  what  the  significance  of  the  difference  is,  or  even  whether  a 
repair  sector  is  high  or  low  risk.  We  compare  individual  evaluation  results 
to  the  expected  results  of  many  similar  evaluations.  (Since  the  methodol¬ 
ogy  developed  in  this  report  is  new,  no  historical  data  exist  for  similar 
evaluations  against  which  to  compare.  We  developed  a  simulation  to  arti¬ 
ficially  create  such  a  background.)  This  comparison  against  expected  re¬ 
sults  permits  us  to  say  whether  either  sector  carries  a  high  or  low  risk.  For 
display  purposes,  we  used  a  combination  of  numerical,  adjectival,  and 
color-coding  ratings  similar  to  those  employed  in  source  selections.9 

♦  Third,  the  literature  on  QFD  offers  little  guidance  on  how  to  elicit  weights 
and  evaluations.  (The  literature  on  MAUT  does  provide  some  guidance.) 


Mathematical  Form  of  the  Model 


Further,  such  judgments  are  known  to  be  subject  to  bias  unless  particular 
attention  is  paid  to  elicitation  methods.10  For  this  reason,  our  repair  sector 
evaluation  procedure  includes  an  evaluation  workbook  and  standardized 
rating  scales. 

Repair  Sector  Selection  Decision 
Mathematical  Model 

A  top-level  view  of  a  small  portion  of  the  QFD-based  repair  sector  selection  deci¬ 
sion  model  is  shown  in  Figure  A-l . 

Figure  A-l.  Quality  Function  Deployment  Model 
for  Evaluating  Alternative  Sources  of  Repair 


♦  The  symbols  at  the  intersections  of  needs  and  characteristics  in  Matrix  A 
represent  strengths  of  interaction  between  needs  and  characteristics — i.e., 
weights — which  determine  the  relative  importance  of  the  various  charac¬ 
teristics.  For  instance,  a  solid  dot  indicates  a  weight  of  “9,”  and  an  open 
dot  indicates  a  weight  of  “1.” 

♦  The  symbols  at  the  intersections  of  characteristics  and  factors  in  Matrix  B 
operate  identically  to  those  in  Matrix  A. 

We  developed  the  repair  sector  evaluation  model  relationships  and  weights  using 
commercial  QFD  software  called  QFD/Capture.11 

Figure  A-2  partly  illustrates  the  internal  logic  of  the  QFD  model.  It  indicates  that 
ability  to  meet  basic  demand  depends  on  capability  and  capacity,  work  stoppage 
protection,  ability  to  reprioritize,  and  customer  linkage  (knowledge  of  the  cus¬ 
tomer’  s  requirements).  Similarly,  capability  and  capacity  to  do  the  work  are  influ¬ 
enced  by  the  extent  to  which  the  work  is  process-dependent,  the  scope  of  the 
relationship  between  the  government  acquiring  agency  and  the  source  of  repair, 
availability  of  more  than  one  source  of  repair,  as  well  as  product  knowledge. 


Figure  A-2.  Tree  Representation  of  the  Model 


Customer  Source  of  repair  Determining  Evaiu- 

needs  x  characteristics  =  x  factors  =  x  ation 


•Meet  basic  deman 

•Agility/responsivene: 

•Reprioritize 

•Surge 

•Adapt  to  new/diff. 
requirements 


•Quality  product  &  service 
•Low  total  cost 
•Of  maintenance 
•Of  administration 
•Of  start-up/capitalization 
•Continuous  imp.  of  product 
&  service 


Capability  &  capaci 
•Scope  &  scale  econ. 
•Overhead  costs 
•Non-value-added  work 
Profits/fees 
Responsiveness 
k  Reprioritization 
Surge 
'Task  flexibility 
Customer  linkage 
•Administrative  ease 
Incentive  to  improve 
•Work  stoppage  protection 
•Pipeline  size 
•Order  &  ship 
•Repair  cycle 
•Material  movement 
•Output  quality 


Nature  of  repair  work 
Relative  size 

•Process  dependence  -189 

Nature  of  relationship 
with  source  of  repair 
•Type  relationship 
•Scope  of  relationship  .315 

•Incentive  structure 

;ature  of  sources  of  repair 
•Availability  of  more  than 
one  source  J>§Z 

•Interest  by  potential 
sources 

•Reserve  capacity 
Customer  knowledge 

Product  knowledge  .441 

•MRP-II  methods 


Overall  score 


.5  2294 

.7  .220 

.7  ,369 

.3  T32 

s  m 


As  described  in  Chapter  2  and  illustrated  here,  determining  factors  may  be  a  func¬ 
tion  of  the  repair  work  itself;  or  may  depend  on  the  nature  of  the  relationship 
between  the  source  of  repair  and  the  government  agency  acquiring  repair  services 
for  the  customer;  or  may  be  directly  related  to  sources  of  repair  themselves. 


A-6 


Mathematical  Form  of  the  Model 


Calculation  on  Figure  A-2  proceeds  from  left  to  right.  If  the  notional  importance 
of  meeting  basic  demand  were  .9  and  the  relationship  weights  are  as  shown  in 
Figure  A-2,  then  the  calculated  factor  weights  (the  relative  importance  of  the 
determining  factors)  are  as  shown  in  Table  A-2. 


Table  A-2.  Example  Calculation  of  Factor  Weights  for  Meeting  Basic  Demand 


Factor 

Calculation 

Result 

(factor  weight) 

Process  dependence 

0.9  x  0.7  x  0.3 

0.189 

Scope  of  relationship 

0.9  x  0.7  x  0.5 

0.315 

Availability  of  more  than  one  source 

0.9  x  0.7  x  0.9 

0.567 

Product  knowledge 

0.9  x  0.7  x  0.7 

0.441 

The  reason  for  calculating  the  relative  importance  of  the  determining  factors  is  to 
provide  a  basis  for  comparing  repair  sectors.  To  illustrate,  suppose  that  the  known 
pragmatic  choices  were  the  following: 

♦  Commercial  sector 

>•  Original  Equipment  Manufacturer  (OEM) 

>-  Third-party  sources  of  repair 

♦  Organic  sector 

>-  A  maintenance  depot  that  has  a  long  history  of  experience  in  similar 
systems. 

The  OEM  probably  has  excellent  product  and  process  knowledge.  An  organic 
source  would  have  some  product  and  process  knowledge  and  would  have  to  be 
provided  additional  information.  Third-party  sources  might  have  limited  initial 
knowledge.  Further,  suppose  that  these  qualitative  comparisons  are  converted  into 
numerical  scores  of  0.9,  0.7,  and  0.5,  respectively.  Then  the  calculations  for  prod¬ 
uct  knowledge  might  be  as  shown  in  Table  A-3. 

Table  A-3.  Example  Factor  Calculations  for  Product  Knowledge 


Source  of  repair 

Product  knowledge 
assessment 

Product 

knowledge 

score 

Product 
knowledge 
factor  weight 

Result  of 
multiplying  score 
and  weight 

OEM 

Excellent 

0.9 

0.441 

0.397 

Organic 

Some  knowledge  has  to 
be  provided 

0.7 

0.441 

0.309 

Third  party 

Limited  knowledge 

0.3 

0.441 

0.132 
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Table  A-3  provided  sample  calculations  for  a  single  factor.  Overall  results  are 
obtained  by  adding  the  results  for  the  different  factors.  Table  A-4  illustrates  for  a 
subset  of  three  factors.  Considering  only  this  subset  of  factors,  the  organic  choice 
would  be  preferred  since  it  has  the  highest  score,  1.312.  (In  this  case,  however,  the 
organic  choice  dominated  both  commercial  sector  possibilities — which  might  not 
always  be  the  situation.) 

Table  A-4.  Example  Factor  Aggregation  for  Three  Factors 


Source 
of  repair 

Results  of  multiplying  factor  scores 
by  factor  weights  for  three  factors 

Total 

score 

(A) 

Scope  of 
relationship 

(B) 

Availability  of  more 
than  one  source 

(C) 

Product 

knowledge 

(A+B+C) 

OEM 

0.443 

0.134 

0.397 

0.974 

Organic 

0.567 

0.436 

0.309 

1.312 

Third  party 

0.220 

0.369 

0.132 

0.721 

The  approach  illustrated  thus  far,  which  follows  the  typical  QFD  “forward”  cal¬ 
culation  flow  from  customer  needs  to  the  evaluation  factors,  provides  an  overall 
grand  total  for  each  of  the  potential  repair  sectors.  It  does  not  reveal  how  well 
each  of  the  original  customer  needs  (i.e.,  meet  basic  demand,  agility/responsive¬ 
ness,  quality  product  and  service, ...)  was  satisfied.  In  the  model  we  provided  that 
capability  as  well. 
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Appendix  B 

Assumed  Distributions  for  Evaluation  Factors 


As  described  in  Chapter  3  of  this  report,  an  approach  to  establishing  significance 
of  a  repair  sector  rating  is  to  recognize  that  if  the  results  of  many  such  ratings 
were  available  then  the  pattern  of  results  would  form  a  backdrop  against  which 
any  individual  rating  could  be  viewed.  However,  the  results  of  many  such  analy¬ 
ses  are  not  available.  Nonetheless,  it  is  possible  to  artificially  create  a  backdrop  of 
results  by  simulating  the  performance  of  a  large  number  of  analyses.  Doing  so  in¬ 
volves  randomly  choosing  input  factors  from  pre-established  “reasonable”  statisti¬ 
cal  distributions  for  each  of  the  evaluation  factors.  This  appendix  documents 
assumed  distributions  for  the  evaluation  factors.  This  appendix  comprises  three 
sections: 

♦  Factors  related  to  the  nature  of  the  work  to  be  performed 

♦  Factors  related  to  the  relationship  with  potential  sources  of  repair 

♦  Factors  related  to  the  nature  of  potential  sources. 

These  sections  (labelled  III,  IV,  and  V)  correspond  to  the  equivalent  sections  in 
the  evaluation  workbook  that  is  also  included  with  this  report  (Appendix  C). 

The  assumed  distributions  in  this  appendix  are  applicable  to  propulsion  workloads 
and  require  modification  for  other  types  of  repair  work.  These  distributions  are  the 
product  of  Logistics  Management  Institute  staff  members  who  have  experience 
with  propulsion  systems  and  depot  maintenance.  Effort  was  made  to  ensure  these 
distributions  are  reasonable  but  it  should  be  recognized  that  they  are  the  product 
of  human  judgment  rather  than  statistical  analysis. 
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Section  III:  Evaluation — Nature  of  Work  To 
Be  Performed 


III-l.  Relative  size  (amount)  of  this  workload  compared  to  total  market.  Includes 
organic  and  private-sector  marketplaces. 


This  workload  is 
the  total  market. 
No  other  similar 
workload  exists 
anywhere. 

(1) 


Total  market 
includes  other 
similar  work  that 
may  be  partially 
commercial.  This 
workload  is  the 
driver — greater 
than  50%. 

(3) 


Total  market  in¬ 
cludes  other 
similar  work  with 
approximately 
equal  proportions 
of  military  and 
commercial  work. 

(5) 


This  workload  is  in  the 
commercial  main¬ 
stream.  Absence  or 
presence  of  this  work¬ 
load  would  have  no 
significant  effect  on  the 
market. 

(9) 
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Assumed  Distributions  for  Evaluation  Factors 


Comments: 

♦  “Typical”  DoD  gas-turbine  engine  depot  workload  is  a  small  amount  (e.g., 
less  than  25  percent)  of  the  worldwide  gas-turbine  engine  depot  workload. 

♦  A  1  rating  is  unlikely  because  all  military  engines  have  some  commonality 
with  other  military  or  commercial  engines. 

♦  A3  rating  is  also  unlikely  because  we  did  not  see  any  cases  where  the 
workload,  by  itself,  would  be  the  driver  of  the  engine  remanufacturing 
market. 

♦  A  9  rating  is  a  small  possibility  because: 

>■  Absence  or  presence  of  the  workload  could  have  a  minor  affect  on  the 
market. 

►  The  majority  of  DoD’s  propulsion  workload  is  close  to  the  commercial 
mainstream  even  though  some  possible  configuration  changes  may 
make  the  engine  difficult  to  support  commercially. 
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III-2.  Relative  size  (amount)  of  this  workload  compared  to  a  typical  source  of  re¬ 
pair’ s  total  workload. 


This  would  be  the 

This  is  approxi- 

This  represents 

SOR’s  only 

mately  one-half  of 

less  than  one- 

workload. 

the  SOR’s  work- 

fourth  of  the 

load. 

SOR’s  workload. 

(i) 

(5) 

(9) 

1 _ 

1 

1 

1 

1 

i — 
i 

i 

3 

1 

5 

1 

7 

1 

9 

Comments: 

♦  The  DoD  workload  is  assumed  to  be  gas-turbine  engine  remanufacturing. 

♦  It  is  unreasonable  to  expect  the  workload  would  be  assigned  to  a  source  of 
repair  that  does  not  currently  accomplish  gas-turbine  remanufacturing 

(a  1  is  therefore  eliminated). 


Assumed  Distributions  for  Evaluation  Factors 


♦  The  resulting  scale  is  bounded  on  the  left  at  3  and  continues  up  to  9 
reflecting  DoD’s  workload  representing  a  range  of  over  one-half  to  less 
than  one-fourth  of  a  typical  source  of  repair’s  total  propulsion  workload. 


B-5 


III-3.  Demand  predictability. 


Forecasting  of 

Forecasting  of 

Has  peaks  and 

No  surprises. 

little  or  no  avail. 

some  use. 

valleys  but 

Forecast  work- 

The  peaks  and 

Workload  will 

amount  of  work- 

load  and  actual 

valleys  of  work- 

vary  from  calen- 

load  consistent 

workload  are 

load  are  extreme. 

dar  quarter  to 

from  one  calen- 

essentially  identi 

The  best  plan  is 

quarter. 

dar  quarter  to 

cal  (e.g.,  time 

flexibility  to  react. 

(5) 

another. 

change  items). 

(i) 

(7) 

(9) 

1 _ l_ 

1 

1 

l 

1 

3 

| 

5 

i 

7 

_ 

9 

Probability 


Comments: 

♦  The  demand  predictability  of  propulsion  workload  is  considered  in  its  en¬ 
tirety — not  the  demand  predictability  of  a  single  engine.  Therefore  the 
greater  the  number  of  engines  in  the  work  package,  the  greater  the  prob¬ 
ability  that  the  peaks  and  valleys  of  individual  engine  remanufacturing  ac¬ 
tions  would  be  evened  out. 

♦  Based  on  the  above,  the  end  anchors  of  1  and  9  are  eliminated  and  the 
mode  is  6. 
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Assumed  Distributions  for  Evaluation  Factors 


III-4.  Design  stability.  Changes  resulting  from  functional  enhancements,  reliabil¬ 
ity  improvements,  etc.,  that  affect  the  repair  process  (e.g.,  change  form,  fit,  or 
function).  Excludes  design  changes  that  do  not  affect  the  repair  process. 


Design  evolving, 
changes  affecting 
the  repair  proc¬ 
ess  a  certainty. 

Some  possibility 
of  changes  that 
affect  the  repair 
process. 

No  changes  envi¬ 
sioned  that  affect 
the  repair  proc¬ 
ess. 

(i) 

(5) 

(9) 

| 

1 

1 

1 

1 

i 

i 

i 

3 

1 

5 

1 

7 

1 

9 

Comments: 

♦  A  center  5  distribution  is  anticipated  because,  for  gas-turbine  remanufac¬ 
turing  workload,  there  is  some  possibly  that  design  changes  may  arise  that 
will  impact  the  repair  process. 

♦  The  anchors  of  1  and  9  are  eliminated  because  there  is  no  absolute  cer¬ 
tainty  that  design  changes  affecting  the  repair  process  will  or  will  not  oc¬ 
cur.  All  other  ratings  are  possibilities. 
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III-5.  Technology  renewal  rate.  How  often  does  one  generation  of  technology  re¬ 
place  another? 


Greater  than  10 
years. 

(1) 


About  5  years. 

(5) 


Less  than  3 
years. 

(9) 


1  3  5  7  9 


Comments: 

♦  The  technology  renewal  rate  for  gas-turbine  engines  is  normally  slow  rela¬ 
tive  to  the  more  electronic/computer  intensive  systems/components.  An 
anchor  of  9  is  therefore  unlikely. 

♦  A  renewal  rate  of  slightly  less  than  10  years  is  considered  the  norm. 
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Assumed  Distributions  for  Evaluation  Factors 


III-6.  Uniqueness  of  technology. 


Unique  military 
technology  with 
few  other  appli¬ 
cations. 

Primarily  military 
technology  but 
many  different 
applications. 

In  mainstream, 
many  applica¬ 
tions  beyond 
military. 

(i) 

(5) 

(9) 
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Comments: 

♦  Gas-turbine  engine  technologies  are  not  military-unique. 

♦  The  majority  of  DoD’s  propulsion  workload  is  close  to  the  commercial 
mainstream  and  a  modal  rating  of  8  is  anticipated. 
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III-7.  Repair  process  dependence.  (Repair  process  includes  skills,  training,  data, 
equipment,  environment,  facilities,  etc.) 


Repair  is  domi- 

Requires  some 

Uses  common, 

nated  by  unique 

unique  proc- 

widely  available 

repair  processes 

esses. 

processes. 

applicable  only  to 

this  workload. 

(5) 

O) 

(D 

_ i _ i _ i _ 

1 

i 

i 

3 

1 

5 

l 

7 

1 

9 

Probability 


Comments: 

♦  Very  few  unique  repair  processes  are  required. 
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Assumed  Distributions  for  Evaluation  Factors 


III-8.  Availability  of  engineering  data  to  design  repair  process.  Does  it  exist? 


Reverse  engi¬ 
neering  required 
to  develop  and 
maintain  repair 
capability. 

(i) 

Reverse  engi¬ 
neering  required 
to  augment  avail¬ 
able  data. 

(5) 

Adequate  data 
available  to  de¬ 
velop  and  main¬ 
tain  repair 
process  without 
reverse  engi¬ 
neering. 

(9) 

L. 
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Probability  -  New  System 
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Probability  -  Existing  System 
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Comments: 

♦  For  newly  acquired  propulsion  systems,  engineering  data  would  always 
exist  at  the  OEM. 

♦  For  existing  systems,  the  likelihood  is  that  engineering  data  exists  (a  mode 
of  7);  however,  reverse  engineering  could  be  required  if,  for  instance,  data 
had  not  been  acquired  or  maintained. 
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Assumed  Distributions  for  Evaluation  Factors 


III-9.  Absence  of  proprietary  data  that  affects  the  design  of  repair  process. 


Proprietary  data 
required — OEM 
(or  subcontractor 
to  OEM)  will 
negotiate  rights 
with  this  source. 

(5) 


1  3  5  7  9 


No  proprietary 
data  impacts. 

(9) 


Proprietary  data 
is  required.  OEM 
(or  subcontractor 
to  OEM)  will  not 
release  proprie¬ 
tary  data  to  this 
source  under 
acceptable  terms. 

(i) 


Comments: 

♦  A  normal  distribution  is  anticipated  for  new  systems  with  both  a  1  and  9 
possible.  The  distribution  is  expected  to  be  centered  at  5. 

♦  An  important  factor  in  determining  whether  the  distribution  shifts  up 
(toward  a  9)  or  down  (toward  a  1)  is  whether  the  likely  source  of  repair  is 
also  the  OEM. 

>-  If  the  OEM  is  also  the  likely  source  of  repair,  fewer  proprietary  im¬ 
pacts  are  anticipated. 
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Section  Iv:  Evaluation — Nature  of 
Relationship  with  Sors 


IV-1.  Ability  to  change  requirements.  Depends  upon  the  flexibility  of  the  relation¬ 
ship  between  buyer  of  repair  services  and  typical  or  likely  source  of  repair.1 


Changing  what,  how 
many,  or  when  is 
difficult  and  time- 
consuming  (typical  of 
firm-fixed-price 
contract). 


Changing  what,  how  many,  or 
when  requires  an  acceptable 
amount  of  time  and  expendi¬ 
ture  of  resources  (typical  of 
basic  ordering  agreement  or 
quarterly  workload 
negotiations). 


Changing  what,  how 
many,  or  when  is  easy 
and  quick  (typical  of 
arrangements  that 
have  equal  sharing  of 
risks  and  benefits). 


a) 


Assumed  Distributions  for  Evaluation  Factors 


Comments: 

♦  Because  a  wide  range  of  formal  and  informal  relationships  can  exist  be¬ 
tween  the  DoD  buyer  of  repair  services  and  a  source  of  repair,  the  full 
range  of  assessments  are  possible. 

♦  Item  managers,  if  asked,  would  probably  assign  a  rating  of  5  anticipating 
expending  time  and  effort  to  make  changes. 

♦  Potential  source’s  of  repair,  if  asked,  would  probably  rate  themselves  a  7 
or  9  believing  they  are  flexible. 
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IV -2.  Scope  of  relationship.  What  is  the  intended  work  package  (how  many  repa¬ 
rable  items  are  in  the  work  package  that  use  complementary  repair  processes  and 
resources)? 


Only  one 

Two  or 

Between  four 

Between  nine 

Twenty  or  more 

reparable 

three 

and  eight  repa- 

and  nineteen 

reparable  items 

item  is  in- 

reparable 

rable  items  are 

reparable  items 

are  included  in 

eluded  in  the 

items  are 

included  in  the 

are  included  in 

the  work  pack- 

work 

included  in 

work  package. 

the  work  pack- 

age. 

package. 

the  work 

age. 

package. 

(5) 

0) 

a) 

(7) 

(3) 

1  3  5  7  9 


Probability 


Comments: 

♦  An  assumption  is  made  that  the  gas-turbine  engine  undergoing  remanu¬ 
facturing  is  disassembled  thus  receiving  benefits  from  vertical  scope  of 
relationship.  For  example,  turbine  blades  are  cleaned,  inspected,  and  pos¬ 
sibly  repaired  using  similar  processes. 

♦  Because  of  the  above  assumption,  the  mode  would  be  an  8. 
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Assumed  Distributions  for  Evaluation  Factors 


IV-3.  Typical  source  of  repair’s  direct  responsibility  for  materiel  management, 
sustaining  engineering,  distribution,  and  transportation. 


Responsibility 
limited  to  repair. 
The  SOR  has  no 
responsibility  for 
nonrepair 
functions. 

(1) 


Responsibility 
includes  repair 
and  some  of  the 
four  related 
functions. 

(5) 


Responsibility 
includes  repair 
and  all  four  re¬ 
lated  functions. 

(9) 


1  3  5  7  9 


Comments: 

♦  Because  of  the  nature  of  propulsion  remanufacturing,  DoD  will  always  as¬ 
sign  some  engineering  responsibilities  to  a  source  of  repair;  therefore  the 
low  anchor  (1)  is  not  anticipated. 

♦  It  is  anticipated  that  the  source  of  repair  will  accomplish  some,  if  not  all, 
of  the  related  functions. 
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IV-4.  Policy  on  reparable  lot 
typical  source  of  repair? 


Workload  accumulates. 
Nothing  is  released  to 
repair  process  until  some 
preset  quantity  of  repa¬ 
rables  are  on  hand. 


(1) 


izes.  What  lot  sizes  are  likely  to  be  presented  to  a 


Monthly  induction 
where  the  quantity 
inducted  is  based  on 
user  needs. 


Every  individual  failure 
in  the  field  triggers  an 
immediate  repair  re¬ 
sponse. 


(5) 


(9) 


Assumed  Distributions  for  Evaluation  Factors 


IV-5.  Procurement  of  repair  parts.  How  much  flexibility  will  the  source  of  repair 
have? 


DoD  materiel 

Exceptions  to  the 

SOR  can  deter- 

management 

DoD  system  require 

mine  best  source, 

system  use  is 

approval.  This  ex- 

best  value. 

mandatory. 

ception  process  will, 

in  fact,  be  used. 

(9) 

(i) 

(5) 

! _ 

1 

I 

1 

1 

1 

1 

3 

1 

5 

1 

7 

i 

9 

Comments: 

♦  A  bi-modal  distribution  of  responses  is  expected. 

>-  If  the  source  of  repair  is  an  organic  depot,  extensive  use  of  the  DoD 
materiel  management  system  is  expected  (although  organic  depots  can 
request  exceptions). 

►  If  the  source  of  repair  is  commercial,  best  source/best  value  could  be 
the  predominant  procurement  policy  dependent  on  terms  and  condi¬ 
tions  of  the  contract. 
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IV-6.  Incentive  structure. 


Provides  limited  in¬ 
centive  to  improve 
cost  and  quality  of 
product  and  service. 
May  encourage  SOR 
to  serve  own  needs 
rather  than  those  of 
customer. 

(i) 


i 


Rewards  and  penal¬ 
ties  tied  only  to  repair 
cost. 

(5) 


3  5 


Provides  rewards  and 
penalties  (e.g.,  profit, 
ability  to  keep  busi¬ 
ness,  accolades,  and 
reprimands).  Direct 
focus  on  cost  and 
quality  of  product  and 
service. 

(9) 


7  9 


Probability 


Comments: 

♦  All  options  except  a  1  are  possible. 

♦  The  mode  is  expected  to  be  6  because  the  majority  of  organic  and  com¬ 
mercial  sources  are  expected  to  have  incentive  policies  in  place  tied  to 
factors  beyond  cost. 
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Assumed  Distributions  for  Evaluations  Factors 


Section  V:  Evaluation — Nature  of  Sources 


V-l.  What  would  be  the  availability  of  alternate  sources  of  repair  if  this  repair 
sector  were  selected? 


Only  one  SOR  will 
exist  (e.g.,  the  OEM  or 
a  single  depot). 

There  will  be  at  least  one 
alternate  SOR. 

Many  alternate  SORs 
will  exist. 

0) 

(5) 

(9) 

1 

_ 1 _ 1 _ L 
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Comments: 

♦  For  new  systems,  viable  responses  are  either  1  or  5  (only  one  source  of 
repair  or  at  least  one  alternate  source  of  repair). 

>-  If  an  OEM  (without  licensed  repair  stations)  is  likely,  a  1  is  antici¬ 
pated. 

>-  However,  if  the  OEM  has  licensed  repair  stations  or  if  an  organic 
source  of  repair  is  chosen,  at  least  one  alternate  source  of  repair  will 
exist. 

♦  For  existing  systems,  a  range  of  responses  from  1  to  9  are  equally  possible. 


B-22 


Assumed  Distributions  for  Evaluations  Factors 


V-2.  Given  this  repair  sector  is  chosen,  what  is  the  likelihood  that  a  source  of  re¬ 
pair  will  have  workload  complementary  to  that  for  which  the  repair  sector  deci¬ 
sion  is  being  made? 


Will  be  a  unique 
workload. 

(1) 


There  will  be  complementary 
workload.  Subject  workload  is 
anticipated  to  comprise  ap¬ 
proximately  50%  of  total  com¬ 
plementary  workload. 

(5) 


Work  will  be  performed 
in  an  integrated  facility. 
Subject  workload  is  one 
of  many  that  use  com¬ 
plementary  resources. 
This  workload  is  less 
than  1 0%  of  total  com¬ 
plementary  workload. 

(9) 


i 


3 


5 


7 
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Comments: 


♦  The  DoD  workload  is  assumed  to  be  gas-turbine  engine  remanufacturing. 

♦  It  is  unreasonable  to  expect  the  workload  would  be  assigned  to  an  SOR 
that  does  not  currently  accomplish  gas-turbine  remanufacturing  (a  1  is 
therefore  eliminated). 
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♦  The  resulting  scale  is  bounded  on  the  left  at  3  and  continues  up  to  9 
reflecting  DoD’s  workload  representing  a  range  of  over  one-half  to  less 
than  one-fourth  of  a  source  of  repair’s  total  propulsion  workload. 
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Probability  -  Existing  System 


Comments: 

♦  For  new  gas-turbine  engines,  at  least  two  sources  of  repair  would  be  ex¬ 
pected — organic  and  the  OEM.  In  addition,  more  than  two  SORs  are  pos¬ 
sible  if  the  OEM  has  licensed  repair  subsidiaries. 

♦  For  existing  engines,  an  organic  source  of  repair  as  well  as  the  OEM  are 
expected  to  be  interested.  In  addition,  additional  potential  sources  would 
be  available  and  would  be  expected  to  respond  to  a  request  for  proposal  or 
similar  instrument. 
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Assumed  Distributions  for  Evaluations  Factors 


V-4.  Reserve  capacity  at  a  potential  source  of  repair  for  this  work. 


Capacity  is  available  to  ac¬ 
complish  anticipated  average 
peacetime  (e.g.,  none-surge) 
workload. 

Capacity  is  available  to 
accomplish  highest  postu¬ 
lated  workload — including 
wartime,  surge  workload. 

(5) 

0) 

-J - 1 - L 

- 1 

123456789 


Comments: 

♦  A  “no  reserve  capacity”  (1)  is  eliminated  because  it  is  not  foreseen  select¬ 
ing  a  source  of  repair  with  no  capacity  to  accomplish  any  of  the  projected 
remanufacturing  workload. 

♦  Alternatives  3  through  9  are  possible;  however,  the  greater  likelihood  is 
that  a  source  of  repair  will  be  selected  that  has  the  reserve  capacity  above 
that  required  to  accomplish  the  anticipated  average  peacetime  workload  (a 
mode  of  6.5  has  been  established). 


V-5.  Typical  source  or  repair’s  capacity  in  excess  of  workload. 


SOR  is  burdened  with 
large  excess  capacity 
that  will  take  more  than 
three  years  to  shed. 
SOR  total  workload,  in¬ 
cluding  this  workload, 
utilizes  less  than  70%  of 
available  capacity. 

(D 


SOR  possesses  some 
excess  capacity.  SOR  total 
workload,  including  this 
workload,  is  approximately 
80%  of  available  capacity. 

(5) 


No  excess  capacity 
above  peak  projected 
workload  for  an  SOR. 
0) 


Assumed  Distributions  for  Evaluations  Factors 


V-6.  With  respect  to  a  typical  SOR,  what  is  the  availability  of  labor  with  relevant 
basic  skills? 


Comments: 

♦  All  ratings  are  possible;  however,  most  sources  would  be  concentrated 
around  a  5  rating  because  they  normally  have  a  core  of  journeymen  techni 
cians  in  place. 


V-7.  Overhead  structure  at  a  potential  source  of  repair. 


Overhead  structure  in¬ 
cludes  charges  from 
engineering,  manufac¬ 
turing,  and  other  activi¬ 
ties  in  addition  to 
remanufacturing/repair. 

(1) 


Overhead  structure  in¬ 
cludes  charges  from  engi¬ 
neering  but  not 
manufacturing. 

(5) 


Overhead  structure 
tied  to  the  specific  re¬ 
quirements  of  the  re¬ 
manufacturing/repair 
workload. 

0) 


Comments: 

♦  All  ratings  are  possible. 

♦  For  propulsion  remanufacturing,  overhead  would  normally  include  some 
engineering  activities. 


Assumed  Distributions  for  Evaluations  Factors 


V-8.  Stability  of  labor  relationships. 


History  of  frequent 
work  stoppages.  Rela¬ 
tionship  with  employ¬ 
ees  makes  future  work 
stoppages  likely. 

(1) 


Some  history  of  work 
stoppages.  Future 
stoppages  probably 
infrequent  and  short 
in  duration. 

No  history  of  work  stop¬ 
pages.  Relationship  with 
employees  makes  work 
stoppages  unlikely. 

(5) 

(9) 

1  3  5  7  9 


Comments: 

♦  It  is  not  likely  that  DoD  propulsion  remanufacturing  workload  would  be 
assigned  to  a  source  of  repair  that  has  an  unstable  labor  relationship  with 
its  employees.  Possible  ratings  of  1  and  3  are  therefore  eliminated. 

♦  It  is  assumed  that  if  the  decision  is  organic,  a  very  stable  relationship 
would  exist. 

♦  It  is  assumed  that  if  the  decision  is  contract,  a  mix  of  stable  and  unstable 
labor  relationships  could  exist. 
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V-9.  Typical  source  of  repair’s  knowledge  of  the  customer  (end  user). 


Recent  and  comparable 
experience  working  with 
end  users  of  comparable 
items. 

Current  and  detailed  knowledge  of 
end  user’s  operational  environment 
and  objectives  affecting  this  item  to 
be  repaired. 

No  experience 
working  with  this  or 
simitar  end  users. 

(D 

(5) 

(9) 

1  3  5  7  9 


Comments: 

♦  It  is  not  likely  that  DoD  propulsion  remanufacturing  workload  would  be 
assigned  to  a  source  of  repair  that  has  little  or  no  experience  working  with 
the  end  user.  Possible  ratings  of  1  and  3  are  therefore  eliminated. 

♦  It  is  assumed  that  if  the  decision  is  organic,  a  high  knowledge  level  of  the 
end  user’ s  operational  environment  would  exist. 

♦  It  is  assumed  that  if  the  decision  is  contract,  a  variety  of  knowledge  levels 
of  the  end  user’s  operational  environment  would  exist. 
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Assumed  Distributions  for  Evaluations  Factors 


V-10.  Typical  source  of  repair’s  knowledge  of  the  specific  component(s)  for 
which  a  repair  sector  decision  is  being  made. 


No  knowledge  of  the 
item  to  be  repaired  and 
has  no  experience  in 
similar  technologies. 

(1) 


Some  knowledge  of  the 
item  to  be  repaired  and 
possesses  experience  with 
similar  equipment  and 
operating  environments. 

(5) 


Demonstrated  knowl¬ 
edge  of  the  item  to  be 
repaired,  interface  with 
the  weapon  system, 
and  its  operating  envi¬ 
ronment. 

(9) 


L 


1 


Probability  -  Existing  System 


Comments: 

♦  It  is  unlikely  that  a  source  of  repair  would  be  selected  (for  either  a  new  or 
existing  system)  that  has  no  knowledge  of  the  item  to  be  repaired.  (A  rat¬ 
ing  of  1  is  therefore  discounted.) 

♦  For  new  propulsion  systems,  the  ratings  would  be  on  the  upper  end  of  the 
scale  reflecting  the  probability  of  workload  assignment  to  an  OEM  or  an 
existing  organic  propulsion  remanufacturing  facility  that  possesses  knowl¬ 
edge  of  the  engine  (or  similar  engines)  requiring  repair. 

♦  For  existing  engines,  the  curve  would  be  expected  to  shift  to  the  left  re¬ 
flecting  the  probability  of  an  increased  number  of  potential  sources,  some 
of  which  would  not  possess  the  degree  of  knowledge  that  the  OEM  and 
organic  depots  possess. 
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Assumed  Distributions  for  Evaluations  Factors 


V-ll.  Source  of  repair’s  knowledge  of  the  specific  remanufacturing/repair  process 
for  which  a  repair  sector  decision  is  being  made. 


Demonstrated  experi- 

No  demonstrated  experi¬ 
ence  with  this  or  compara¬ 
ble  processes. 

Demonstrated  experience 
with  comparable  complex¬ 
ity  processes. 

ence  with  required 
process  and  ability  to 
optimize  the  process  in 
specific  applications. 

(i) 

(5) 

0) 

Comments: 

♦  It  is  unlikely  that  a  source  of  repair  would  be  selected  (for  either  a  new  or 
existing  system)  that  has  no  knowledge  of  the  remanufacturing/repair 
process.  (Possible  ratings  of  1  and  3  are  therefore  discounted.) 

♦  For  new  propulsion  systems,  the  ratings  would  be  on  the  upper  end  of  the 
scale  reflecting  the  high  probability  of  workload  assignment  to  an  OEM  or 
an  existing  organic  propulsion  remanufacturing  facility. 

♦  For  existing  propulsion  systems,  the  ratings  are  similar  because,  in  addi¬ 
tion  to  the  possibility  of  workload  assignment  to  an  OEM  or  an  existing 
organic  propulsion  remanufacturing  facility,  third-party  sources  have  had 
the  opportunity  to  become  familiar  with  the  repair  process. 
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Assumed  Distributions  for  Evaluations  Factors 


V-12.  Typical  or  anticipated  source  of  repair’s  use  of  integrated  manufacturing, 
engineering,  financial  planning  and  execution  MRP-II  management  methods,  and 
systems. 


MRP-II  consistent 
methods  and  sys¬ 
tems. 

(5) 


1  3  5  7  9 


Pre-MRP-ll  methods 
and  systems. 

(1) 


Advanced  with  MRP- 
II/Enterprise  Resource 
Planning  (ERP)  meth¬ 
ods  and  systems. 

(9) 


Comments: 

♦  Based  on  the  assumption  that  both  organic  and  commercial  sources  are 
moving  in  the  direction  of  institutionalizing  MRP-II  methods,  a  rating  of  1 
is  not  considered  likely. 

♦  Looking  out  at  the  next  3  to  7  years,  the  most  likely  probability  would 
range  between  a  5  and  7,  centered  on  a  6. 
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Appendix  C 

Trial  Evaluation  Workbook 


Chapter  3  described  the  use  of  a  workbook  in  support  of  integrated  prod¬ 
uct/process  teams  as  they  evaluate  repair  sectors  in  terms  of  the  factors  described 
in  Chapter  2.  This  appendix  provides  an  example  evaluation  workbook.  The 
workbook  provided  in  this  appendix  was  tailored  for  complex  avionics  workloads 
and  is  an  updated  version  of  the  workbook  used  to  assist  in  deciding  which  repair 
sector  should  support  the  V-22  forward  looking  infrared  (FLIR)  sub-system. 


Repair  Sector  Selection  Workbook 

Introduction 
Version  4. 1 


DEPOT  MAINTENANCE 
REPAIR  SECTOR  SELECTION 
METHODOLOGY 


WORKBOOK 


Repair  Sector  Selection  Workbook 

Introduction 
Version  4. 1 

This  Workbook  is  intended  to  assist  in  determining  which  repair 
sector  (public  or  commercial)  is  the  best  choice  for  repair  of  the 
V- 22  Forward  Looking  Infrared  (FLIR)  sub-system. 

The  selection  of  a  repair  sector  is  critical  for  each  component  or  system.  Obviously,  it  is 
vital  to  select  a  repair  sector  that  can  satisfactorily  accomplish  the  maintenance  within 
required  time-frames,  with  required  quality,  and  do  so  economically. 

Selecting  the  “right”  repair  sector  involves  balancing  characteristics  of  the  system  to  be 
maintained  and  prospective  repair  sectors  against  user  needs.  The  method  used  to 
attain  this  balance  has  been  shown  effective  in  industry  for  customer  focused  process 
and  product  design  and  is  called  Quality  Function  Deployment  (QFD).  QFD  is  a  system 
for  listening  to  the  “voice  of  the  customer”  to  bringing  customers  and  providers  together 
through  an  intensive,  interactive  process. 


The  QFD  process  as  applied  here  is  shown  graphically  in  figure  1.  Customer  inputs 
(user  needs)  which  provide  the  measure  of  “quality”  in  the  process,  are  the 

Repair  sectors  and  their  characteristics 


Influencing  Factors 

Matrix  “B” 


starting  point  for  the  QFD  matrices.  These  prioritized  operational  customer  needs 
(responsiveness,  low  cost,  etc.)  are  incorporated  in  Matrix  A  on  the  left  side  of  figure  1 . 
Quality  is  “deployed”  from  one  matrix  to  the  other  sequential  manner  (Matrix  A  to  Matrix 
B)  as  indicated  by  the  arrow.  The  QFD  matrices  record  the  interrelationships  between 
needs,  repair  sector  characteristics,  and  influencing  factors  as  well  as  the  “weights”  or 
strengths  of  these  relationships.  Lists  of  the  repair  sector  characteristics  and  influencing 
factors  are  provided  in  Sections  I  and  II. 
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Repair  Sector  Selection  Workbook 

Introduction 
Version  4. 1 


This  Workbook  assists  logistics  planners  in  rating  each  of  the  influencing  factors  in 
Matrix  B  on  a  scale  of  1  to  9.  The  evaluation  results  will  then  be  transferred  to  the  QFD 
model  and  the  model  will  do  the  mathematics  to  combine  these  evaluations  with  the 
operational  needs  input  to  complete  the  analysis.  The  influencing  factors  you  will  be 
evaluating  in  the  Workbook  are  grouped  in  the  following  categories: 

•  Nature  of  the  work  to  be  performed 

•  Nature  of  relationship  with  sources  of  repair 

•  Nature  of  potential  sources 

Sections  III,  IV,  and  V  form  the  heart  of  the  workbook  and  are  where  you  are  requested 
to  evaluate  the  Influencing  Factors.  Each  factor  is  listed  followed  by  a  narrative 
description  of  a  poor  assessment,  nominal  assessment,  and  best  assessment.  A  scale 
containing  numerical  values  of  1,  3,  5,  7,  and  9  is  provided.  You  are  requested  to 
record  the  number  that  best  characterizes  each  determining  factor. 

Your  care  and  thoroughness  in  completing  this  workbook  will  be  appreciated. 
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Repair  Sector  Selection  Workbook 
Section  I  -  SOR  Essential  Attributes 

Version  4. 1 


SECTION  I 

SOURCE  OF  REPAIR  CHARACTERISTICS 


ADEQUATE  CAPABILITY  &  CAPACITY 
SCOPE  AND  SCALE  ECONOMY 
OVERHEAD  COSTS 
NON-VALUE-ADDED  EFFORT 
EXTRAORDINARY  PROFITS,  FEES,  EARNINGS 
WORK  STOPPAGE  PROTECTION 

RESPONSIVENESS  TO  CHANGING  NEEDS 

•  Reprioritization  Capability 

•  Surge  Capability 

•  Task  Flexibility 

•  Customer  Linkage 

•  Administrative  Ease 

INCENTIVE  TO  IMPROVE 

PIPELINE  SIZE  (QUANTITY  AND  TIME) 

•  Order  and  Ship  (Administrative) 

•  Repair  Cycle  at  SOR 

•  Material  Movement  (User-SOR-User) 

OUTPUT  QUALITY 

•  Of  Product 

•  Of  Service 
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Repair  Sector  Selection  Workbook 
Section  II  -  Influencing  Factors 

Version  4. 1 


SECTION  II 

INFLUENCING  FACTORS 


NATURE  OF  WORK  TO  BE  PERFORMED 

•  Relative  size  (amount)  compared  to  market  place 

•  Relative  size  (amount)  compared  to  SORs  total  workload 

•  Demand  predictability 

•  Design  stability 

•  Technology  renewal  rate 

•  Uniqueness  of  technology 

•  Repair  process  dependence 

•  Availability  of  engineering  data 

•  Absence  of  proprietary  data 

NATURE  OF  RELATIONSHIP  WITH  SORs 

•  Ability  to  change  requirements 

•  Scope  of  relationship 

•  SOR’s  responsibility  for  related  functions  (e.g.,  engineering) 

•  Policy  on  reparable  lot  sizes 

•  Policy  on  procurement  of  repair  parts 

•  Incentive  structure 

NATURE  OF  SOURCES 

•  Availability  of  more  than  one  qualified  source 

•  Availability  of  sources  with  complementary  workload 

•  Interest  by  potential  sources 

•  Reserve  capacity  for  this  work 

•  Capacity  in  excess  of  workload 

•  Availability  of  skilled  labor 

•  Overhead  structure 

•  Stability  of  SOR’s  labor  relationship 

•  Customer  knowledge 

•  Product  knowledge 

•  Process  knowledge 

•  MRP-II  consistent  management  methods 
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Repair  Sector  Selection  Workbook 
Section  III  -  Nature  of  Work  to  be  Performed 

Version  4. 1 


Evaluation  Factor:  111-1 . 

Relative  size  (amount)  of  this  workload  compared  to  total  market.  Includes  organic  and  private  sector 
market  places. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

0 

O 

0 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material  movement 
(user-SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

Comments: 
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Repair  Sector  Selection  Workbook 
Section  III  -  Nature  of  Work  to  be  Performed 

Version  4. 1 


Evaluation  Factor:  111-2. 

Relative  size  (amount)  of  this  workload  compared  to  typical  commercial  or  organic  source  of  repair’s  total 
workload. 

This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

U 

LJ 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material  movement 
(user-SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

w 

Evaluation: 


P°or  Nominal  Best 

Assessment  Assessment  Assessment 


Enter  contract  score  in  this  block 

_ 1 

Enter  organic  score  in  this  block 

Comments: 
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Repair  Sector  Selection  Workbook 
Section  lil  -  Nature  of  Work  to  be  Performed 

Version  4. 1 


Evaluation  Factor:  111-3. 

Demand  predictability1. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

• 

Comments: 


1  Evidence  is  that  most  avionics  would  rate  a  "1"  on  the  above  scale  because  predictability  of  demand  for 
modern  avionics  is  poor  in  general.  This  means  that  in  order  for  a  source  of  repair  to  see  a  reasonably 
steady  workload,  a  relatively  large  number  of  complimentary  units  (i.e.,  units  that  use  essentially  the  same 
repair  process)  need  to  be  "pooled."  The  problem  is  less  severe  with  mechanical  components. 
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Section  III  -  Nature  of  Work  to  be  Performed 

Version  4. 1 


Evaluation  Factor:  111-4. 

Design  stability.  Changes  resulting  from  functional  enhancements,  reliability  improvements,  etc.,  that 
affect  the  repair  process  (e.g.,  change  form,  fit,  or  function).  Excludes  design  changes  that  do  not 
affect  the  repair  process. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead 

costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

• 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material  movement 
(user-SOR-user) 

Output  quality  of 
product 

Output  quality 
of  services 

0| 

y 

0 

i 

[Evaluation: 


Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Section  III  -  Nature  of  Work  to  be  Performed 

Version  4. 1 


Evaluation  Factor:  111-5. 

Technology  renewal  rate.  How  often  does  one  generation  of  technology  replace  another? 

This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead 

costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

9 

0 

0 

0 

y 

Evaluation: 


Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  &  organic  score  in  this  block 


Comments: 
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Repair  Sector  Selection  Workbook 
Section  HI  -  Nature  of  Work  to  be  Performed 

Version  4. 1 

Evaluation  Factor:  111-6. 

Uniqueness  of  technology. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness 
to  changing 
needs: 

Surge  capability 

Responsiveness 
to  changing 
needs: 

Task  flexibility 

w 

• 

(3 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

i 

Output  quality 
of  services 

Evaluation: 


Poor  Assessment  Nominal  Best 

Assessment  Assessment 

Enter  contract  &  organic  score  in  this  block  | 

Comments: 
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Evaluation  Factor:  111-7. 

Repair  process  dependence.  (Repair  process  includes  skills,  training,  data,  equipment,  environment, 
facilities,  etc.) 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead 

costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

© 

• 

0 

0 

0 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

Evaluation  Scoring: 


Nominal  Best 

Assessment  Assessment 


Enter  contract  &  organic  score  in  this  block 


Comments: 


C-15 


Repair  Sector  Selection  Workbook 
Section  III  -  Nature  of  Work  to  be  Performed 

Version  4. 1 

Evaluation  Factor:  111-8. 

Availability  of  engineering  data  to  design  the  repair  process.  Does  it  exist? 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead 

costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness 
to  changing 
needs: 

Surge  capability 

Responsiveness 
to  changing 
needs: 

Task  flexibility 

© 

0 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

;  movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

0 

Evaluation: 


Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  &  organic  score  in  this  block 


Comments: 
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Evaluation  Factor:  111-9. 

Absence  of  proprietary  data  that  affects  the  design  of  repair  process. 

This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

LJ 

0 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

(Evaluation: 


Poor  Assessment 


Nominal 

Assessment 


Best  Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  IV-1. 

Ability  to  change  requirements.  Depends  upon  the  flexibility  of  the  relationship  between  buyer  of 
repair  services  and  source  of  repair1. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

(3 

a 

□ 

U 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

w 

9 

• 

9 

0 

0 

0 

0 

‘Evaluation: 


Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 


1  Basis  of  relationship  may  be  formal  or  informal  e.g.,  firm  fixed  price  contract,  basic  ordering  agreement  (BOA), 
memorandum  of  agreement  (MOA),  organic  production  schedule,  etc. 
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Evaluation  Factor:  IV-2. 

Scope  of  relationship1.  What  is  the  intended  work  package  (how  many  reparable  items2  are  in  the 
work  package  that  use  complementary  repair  processes  and  resources)? 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead 

costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

n 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

Evaluation: 


Poor  Assessment  Nomina!  Best  Assessment 

Assessment 


Enter  contract  &  organic  score  in  this  block 


Comments: 


1  The  scope  of  relationship  is  important  to  workload  leveling.  The  broader  the  scope,  the  more  peaks  and  valleys 
from  individual  components  will  level  each  other  out.  Complementary  processes  and  resources  use  similar 

(or  the  same)  skills,  similar  (or  the  same)  equipment,  similar  (or  the  same)  task  sequences,  etc.  Scope  can  be 
either  vertical  (many  components  from  the  same  system)  or  horizontal  (similar  components  from  many  systems). 

2  Examples  of  reparable  items  include  actuators,  engine  components,  avionics  components,  landing  gears, 
and  transmission  assemblies. 
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Evaluation  Factor:  IV-3. 

Source  of  repair's  direct  responsibility  for:  materiel  management,  sustaining  engineering,  distribution  and 
transportation. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead 

costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

i  Pipeline  size 
Material  movement 
(user-SOR-user) 

Output  quality 
of  product 

Output  quality 
of  sen/ices 

0 

w 

[Evaluation: 


Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

_ 1 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  IV-4. 

Policy  on  reparable  lot  sizes.  What  lot  sizes  will  be  presented  to  the  source  of  repair? 
This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness 
to  changing 
needs: 

Task  flexibility 

□ 

© 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

• 

0 

Evaluation: 


1  23  456789 

Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  IV-5. 

Procurement  of  repair  parts.  How  much  flexibility  will  the  source  of  repair  have? 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead 

costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

• 

0 

0 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

□ 

Evaluation: 


Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  factor:  IV-6. 

Incentive  structure. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead 

costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing 
needs: 

Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

0 

O 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

□ 

9 

□ 

Evaluation: 


Provides  limited  incentive  to 
improve  cost  and  quality  of 
product  and  service.  May 
encourage  SOR  to  serve  own 
needs  rather  than  those  of 
customer. 

in _ 

- 1 - \ 

1  2  3 


Rewards  and 
penalties  tied  only  to 
repair  cost. 

(5) 

I 


4  5 


6 


Provides  rewards  and  penalties 
(e.g.,  profit,  ability  to  keep  business, 
accolades  and  reprimands).  Direct 
focus  on  cost  and  quality  of  product 
and  service 
0) 


7  8  9 


Poor  Nominal  Best  Assessment 

Assessment  Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  V-1. 

What  would  be  the  availability  of  alternate  sources  of  repair  if  this  repair  sector  were  selected? 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead 

costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

© 

© 

O 

□ 

(3 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

4 

Evaluation: 


Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

- 1 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  V-2. 

Given  this  repair  sector  is  chosen,  what  is  the  likelihood  that  a  typical  source  of  repair  will  have  workload 
complementary  to  that  for  which  the  repair  sector  decision  is  being  made?1 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

□ 

0 

© 

• 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

[Evaluation: 


Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 


1  Complementary  workload  is  important  to  workload  leveling.  The  greater  the  amount  of  complementary 
workload,  the  more  peaks  and  valleys  from  individual  components  will  level  each  other  out. 
Complementary  processes  and  resources  use  similar  (or  the  same)  skills,  similar  (or  the  same) 
equipment,  similar  (or  the  same)  task  sequences,  etc.  Complementary  workload  can  be  either  vertical 
(many  components  from  the  same  system)  or  horizontal  (similar  components  from  many  systems). 
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Evaluation  Factor:  V-3. 

Interest  in  this  workload  by  potential  sources. 

This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non- value-added 
effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness 
to  changing 
needs: 

Task  flexibility 

0 

0 

0 

Responsiveness  to  , 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

0 

{Evaluation: 


123  456789 

Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  V-4. 

Reserve  capacity  at  a  potential  source  of  repair  for  this  work. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

O 

LJ 

El 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

Evaluation: 


Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  V-5. 

Typical  source  of  repair's  capacity  in  excess  of  workload. 


Evaluation: 


SOR  is  burdened  with  large  excess 
capacity  which  will  take  more  than 
three  years  to  shed.  SOR  total 
workload,  including  this  workload, 
utilizes  less  than  70%  of  available 
capacity. 

(1) 


SOR  possesses  some 
excess  capacity.  SOR  total 
workload,  including  this 
workload,  is  approximately 
80%  of  available  capacity. 
(5) 


No  excess  capacity 
above  peak  projected 
workload  for  this  SOR. 
(9) 


Poor  Assessment 


Nominal 

Assessment 


Best  Assessment 


Enter  contract  score  in  this  block 


Enter  organic  score  in  this  block 


Comments: 
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Evaluation  Factor:  V-6. 

With  respect  to  a  typical  source  of  repair,  what  is  the  availability  of  labor  with  relevant  basic  skills? 

This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

El 

© 

• 

o 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

9 

Evaluation: 


123456789 

Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  V-7. 

Overhead  structure  at  a  potential  source  of  repair. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value- added 
effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

□ 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

Comments: 
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Evaluation  Factor:  V-8. 

Stability  of  labor  relationships. 


This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

0 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
S  OR- user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

0 

0 

Evaluation: 


Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

- 1 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  V-9. 

Typical  source  or  repair's  knowledge  of  the  customer  (end  user). 

This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

0 

w 

Q 

9 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

• 

w 

9 

ca 

LJ 

Comments: 
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Evaluation  Factor:  V-10. 

Typical  source  of  repair's  knowledge  of  the  specific  component(s)  for  which  a  repair  sector  decision 
is  being  made. 

This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

(3 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

Q 

[Evaluation: 


Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  V-11. 

Source  of  repair's  knowledge  of  the  specific  remanufacturing/repair  process  for  which  a  repair  sector 
decision  is  being  made. 

This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

© 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material  movement 
(user-SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

□ 

LJ 

Evaluation: 


Poor  Assessment  Nominal  Best  Assessment 

Assessment 


Enter  contract  score  in  this  block 

Enter  organic  score  in  this  block 

Comments: 
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Evaluation  Factor:  V-12. 

Typical  source  of  repair's  use  of  integrated  manufacturing,  engineering,  financial  planning  and  execution 
[Manufacturing  Resources  Planning  (MRP-II)]  management  methods  and  systems. 

This  factor  affects: 


Adequate  Capacity 
and  Capability 

Scope  &  scale 
economy 

Overhead  costs 

Non-value-added 

effort 

Extraordinary 
profits,  fees, 
earnings 

Work  stoppage 
protection 

Responsiveness  to 
changing  needs: 
Reprioritization 
capability 

Responsiveness  to 
changing  needs: 
Surge  capability 

Responsiveness  to 
changing  needs: 
Task  flexibility 

□ 

0 

Responsiveness  to 
changing  needs: 
Customer  linkage 

Incentive  to 
improve 

Administrative 

ease 

Pipeline  size 
Order  and  ship 
(admin.) 

Pipeline  size 
Repair  cycle  at 
SOR 

Pipeline  size 
Material 

movement  (user- 
SOR-user) 

Output  quality 
of  product 

Output  quality 
of  services 

0 

• 

0 

0 

Evaluation: 

Advanced  with  MRP- 
II/Enterprise  Resource 
Planning  (ERP) 
methods  and  systems.3 
(9) 

Pre  MRP  methods 
and  systems.1 

(1) 

MRP  consistent 
methods  and 
systems. 2 
_ (5) _ 

1 _ 

1  1  1 

| 

1 

1  1 

| 

1 

1 

1  1  1 

2  3  4 

1 

5 

1 

6 

1 - 1 — 

7  8 

- 1 

9 

Poor  Assessment 

Nominal 

Assessment 

Best  Assessment 

Enter  contract  score  in  this  block 

|Enter  organic  score  in  this  block| 

Comments: 


1  Production/remanufacturing  resource  planning  processes  are  based  on  historical  usage  and/or  one  time  buys  for  items.  Purchasing  and 
manufacturing  orders  of  individual  parts  and  assemblies  occur  when  predetermined  order  points  are  reached  or  parts  are  simply  ordered  as 
a  need  is  identified.  Automated  support  may  exist,  but  it  does  not  support  time-phased  acquisition  and  assembly  of  components, 
subassemblies  and/or  end  items  with  closed  loop  feedback  between  remanufacturing,  finance,  and  customers. 

2  Production/remanufacturing  resource  planning  system  (homegrown  or  commercial  planning  system)  supports  multi-level  bills  of 
material  &  routers  construction  for  the  generation  of  time-phased  purchasing  and  production  requirements  from  a  master  production 
schedule  (MPS).  The  MPS  is  input  into  a  material  requirements  or  finite  scheduling  planning  processor.  The  output  of  the 
production/manufacturing  resources  planning  system  is  used  to  assess  capacity,  plan  &  control  material  orders,  shop  orders, 

and  to  provide  exception  messages  which  require  follow-up  and  correction.  The  automated  planning  system  only  partially  supports  the 
management  of  the  business  and  does  not  fully  support  a  closed  loop  feedback  among  remanufacturing,  engineering,  finance  and  customers. 

3  Production/remanufacturing  resource  planning  system  is  a  closed  loop  system,  providing  planning,  control  and  measurement  support  for  the 
management  of  the  entire  business  enterprise.  Inventory  bills  of  material  and  routers  accuracy  are  at  least  95%  or  better.  It  provides  near 
real  time  feedback  on  execution  versus  the  plan  and  inventory  status  to  finance,  customer  interface/marketing,  and  remanufacturing 
operations.  The  automated  information  system  includes  interfaces  and  integrated  systems  support  for  all  enterprise  related  functions 
including  engineering,  human  resources,  quality  etc.  and  may  additionally  dynamically  link  to  external  customer  and  supplier  chain  systems. 
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Appendix  D 

Factors  Retained  in  the  Model 


The  three  tables  in  this  appendix  indicate  which  factors  should  be  retained  or 
eliminated  based  on  the  keep-discard  criteria  established  in  Chapter  4.  The  second 
and  third  columns  in  these  tables  indicate  our  keep  or  discard  recommendations. 
The  fourth  column  is  importance  to  customer  needs  as  computed  by  the  repair 
sector  model,  based  on  the  relationship  weights  as  assigned  by  LMI  staff  mem¬ 
bers.  (This  computed  importance  is  independent  of  any  actual  repair  sector 
evaluation.)  The  next  column  is  the  ordinal  rank  of  each  factor  based  on  the  com¬ 
puted  importance.  As  examples,  the  first  and  second  ranked  factors  are  “incentive 
structure”  (Table  4-4)  and  “customer  knowledge”  (Table  4-5).  The  lowest-ranked 
factor  was  “overhead  structure”  (Table  4-5).  The  sixth  column  indicates  whether 
the  factor  evaluation  depended  on  the  repair  sector  being  evaluated.  As  an  exam¬ 
ple,  “technology  renewal  rate”  and  “uniqueness  of  technology”  (both  in  Table  4-3) 
do  not.  The  next  column  indicates  whether  the  evaluators  understood  the  construct 
represented  by  the  factor.  LMI  facilitators  made  this  judgment.  The  two  factors 
where  problems  persisted,  even  after  sharpening  the  factor  definitions  and  an¬ 
chors,  were  “reserve  capacity  for  this  work”  and  “capacity  in  excess  of  workload” 
(both  in  Table  4-5).  The  final  column  in  each  table  explains  our  recommendation 
to  retain  or  eliminate  the  factor. 
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Table  D-l.  Evaluation  Factors  Retained  for  Nature  of  Work 


Factors  Retained  in  the  Model 
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Table  D-2.  Evaluation  Factors  Retained  for  Nature  of  Relationship 


Factors  Retained  in  the  Model 
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Table  D-3.  Evaluation  Factors  Retained  for  Nature  of  Repair  Sector 


Factors  Retained  in  the  Model 
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Appendix  E 

Abbreviations 


AHP 

analytic  hierarchy  process 

CCFD 

Cockpit  Control  Feel  and  Drive 

DTA 

decision  tree  analysis 

FLIR 

forward-looking  infrared 

IPT 

integrated  process  team 

LCC 

life-cycle  cost 

LMI 

Logistics  Management  Institute 

MAUT 

multiattribute  utility  theory 

MRP-II 

manufacturing  resources  planning 

OEM 

original  equipment  manufacturer 

QFD 

quality  function  deployment 

RSS 

repair  sector  selection 

SOR 

source  of  repair 

SPO 

system  program  office 
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